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Abstract 

Significant  technological  advances  over  the  past  few  decades  have  fueled  the 
continual  and  rapid  development  of  an  information-based  world.  Network  Centric 
Warfare  (NCW)  has  become  the  buzzword  of  the  young  millennium  within  the 
Department  of  Defense  (DoD)  and  is  quickly  becoming  a  popularly  shared  vision  and 
rallying  cry  for  force  transfonnation  among  United  States  military  leaders.  An  essential 
element  in  fully  implementing  this  network-centric  way  of  thinking  is  to  develop  useful 
measures  to  help  gauge  the  effectiveness  and  efficiency  of  both  our  military  networks  and 
our  strategic  NCW  doctrine.  The  goal  of  this  research  is  first  to  provide  a  comprehensive 
summary  of  the  key  literary  works  that  have  forged  a  foundational  basis  for  defining 
NCW.  Second,  this  work  will  utilize  a  System  Effectiveness  Analysis  Simulation 
(SEAS)  combat  model,  which  represents  a  Kosovo-like  engagement  (provided  by  the 
Space  and  Missile  Center),  to  serve  as  a  tool  in  exploring  the  use  of  NCW  metrics  in 
military  worth  analysis.  Third  and  last,  this  effort  selects  measures  for  the  physical, 
information,  and  cognitive  domains  of  NCW  and  analyzes  the  outputs  from  the  Kosovo 
scenario  that  are  pertinent  to  each  domain  in  order  to  assess  the  usefulness  of  each  metric. 
In  the  final  analysis,  the  average  target  detection  distance  outputs  and  average 
communication  channel  message  loading  metrics  chosen  for  the  physical  and  information 
domains  yielded  mixed  results  and  levels  of  utility,  while  the  highly  aggregated  metric  of 
target  kills  served  as  a  useful,  and  yet  rough,  final  metric  for  the  cognitive  domain. 
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MODELING  NETWORK  CENTRIC  WARFARE  (NCW)  WITH 
THE  SYSTEM  EFFECTIVENESS  ANALYSIS  SIMULATION  (SEAS) 

I.  Introduction 

1.1  Background 

In  the  current  Information  Age,  success  or  failure  of  operations  often  relies 
heavily  on  the  ability  to  gather,  translate,  and  process  large  amount  of  data  and 
information.  Evidence  of  this  phenomenon  can  clearly  be  seen  within  two  distinct 
environments:  the  American  business  arena  and  the  World  Wide  Web.  In  the  American 
business  arena,  Wal-Mart  has  moved  from  a  traditional  retailer  to  a  precision  retailer  by 
achieving  information  superiority  in  its  domain  (Alberts,  Gartska,  and  Stein,  1999:46). 
The  end  result  of  Wal-Mart’s  highly  network  and  information-focused  approach  to  retail 
sales  is  that  its  stores  reign  as  the  nation’s  top  retailer,  having  $256  billion  in  annual  sales 
for  2004  (Wal-Mart,  2005:2).  As  for  the  World  Wide  Web,  the  multitude  of  applications 
for  networking  and  sharing  information  on  a  global  scale  continue  to  be  developed  and 
applied. 

The  combination  of  digital  communications  capabilities  and  breakthroughs  in 
software  technology  in  the  form  of  Web  browsers  and  servers  has  combined  to 
enable  information  interactions  among  entities  of  virtually  any  size  that  can  be 
connected  to  the  Internet.  The  net  result  is  referred  to  by  some  as  the  social- 
technological  phenomenon,  the  “Internet  Tsunami”.  (Alberts,  Gartska,  and  Stein, 
1999:250) 

The  same  principles  of  infonnation  dominance  and  power  which  have 
transfonned  the  U.S.  market  place  and  linked  the  world  via  the  internet  apply  equally  to 
the  United  States  military.  Information  technology  has  significantly  changed  our 


1 


concepts  of  time  and  distance.  Distance  is  becoming  less  relevant  as  large  amounts  of 
information  are  able  to  be  transmitted  and  received  with  increasing  ease  and  speed. 
Within  the  battlespace,  this  shrinking  of  distance  and  time  translates  into  increased 
combat  capability  and  the  potential  for  orders  of  magnitude  increases  in  mission 
effectiveness  and  efficiency.  The  key  to  realizing  this  potential  is  the  ability  to  fully 
utilize  our  systems  of  sensors,  data  processors,  communication  links,  and  decision¬ 
making  methods. 

A  ground-breaking  concept  that  moves  the  U.S.  military  towards  the  goal  of 
achieving  maximum  combat  success  and  efficiency  through  utilization  of  network 
technology  has  emerged  over  the  last  five  to  ten  years.  This  revolutionary  idea  is  called 
Network  Centric  Warfare  (NCW).  The  defining  characteristics  and  exact  applications  of 
NCW  are  continually  evolving,  as  are  its  applications.  NCW  finds  itself  being  explored 
and  studied  as  part  of  a  larger  initiative  within  the  DoD,  that  of  transfonnation.  A 
primary  goal  of  transformation,  and  consequently  of  the  Office  of  Force  Transformation, 
is  to  keep  the  United  States  military  at  the  forefront  of  warfare  technology,  tactics,  and 
knowledge  of  the  enemy.  The  concept  of  effects-based  operations  (EBO)  is  being 
employed  as  a  theoretical  means  to  the  end  of  military  transformation.  A  policy  of 
“forward  deterrence”  has  been  adopted  by  the  Office  of  Force  Transfonnation  and  NCW 
figures  to  play  a  key  role  in  implementing  this  policy. 

1.2  Problem  Definition 

The  technological  advances  of  the  Information  Age  have  not  only  increased 
capacities  of  information  exchange  and  decreased  information  processing  time,  but  have 
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also  increased  levels  of  complexity  involved  with  sorting  through  data  and  information  to 
find  the  packets  that  are  pertinent  to  a  certain  decision  or  problem.  The  higher  levels  of 
complexity  involved  in  vast  information  networks  and  systems  make  it  difficult  to  assess 
the  relative  worth  and  efficiency  of  these  networks  and  systems.  The  development  of 
basic,  definable,  and  measurable  metrics  is  required  in  order  to  serve  as  diagnostic  tools 
for  rating  the  effectiveness  of  network  perfonnance  and  impact  on  command  and  control, 
especially  within  a  military  system  or  tactical  engagement.  These  measures  will  be 
determined  and  chosen  based  on  available  outputs  from  a  combat  modeling  scenario. 
Once  basic  metrics  are  established,  their  military  worth  can  be  measured  through  the 
utilization  of  various  analysis  methods  and  tools  applied  to  output  data  from  multiple 
combat  simulation  runs.  In  short,  the  essence  of  the  problem  for  this  thesis  effort  is  to 
determine  how  to  measure  the  effective  application  and  worth  of  NCW  within  the  context 
of  a  combat  modeling  simulation. 

1.3  Research  Objective 

This  research  has  been  sponsored  by  the  Simulation  and  Analysis  Facility 
(SIMAF)  at  Wright-Patterson  AFB,  OH.  This  effort  is  focused  on  first  defining  Network 
Centric  Warfare  (NCW)  from  reputable  research  literature  and  doctrinal  documents 
pertaining  to  the  subject.  This  definition  will  be  fonned  against  the  back-drop  of  the 
larger  picture  of  force  transformation  currently  being  employed  within  the  Department  of 
Defense.  Once  defined,  various  modeling  techniques  and  metrics  for  NCW  will  be 
addressed  and  established.  From  these  proposed  models  and  metrics,  a  specific  modeling 
option  will  be  chosen  and  utilized  in  order  to  measure  the  military  worth  of  NCW  in  a 
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well  defined  mission  level  scenario.  The  focus  of  analysis  will  be  primarily  on 
contrasting  the  performance  of  an  NCW-enabled  force  in  a  given  combat  situation  versus 
the  perfonnance  of  that  same  force  acting  at  degraded  levels  of  NCW  capability.  The 
outputs  resulting  from  the  baseline  case  and  NCW  degraded  cases  will  be  analyzed  to 
provide  insight  into  the  benefits  and  challenges  of  utilizing  NCW  as  an  applied  theory  of 
conducting  military  operations. 

1.4  Research  Scope 

For  this  effort,  Modeling  and  Simulation  (M&S)  will  be  the  primary  specialty 
within  the  Operations  Research  tool  set  that  will  be  utilized  as  a  means  of  evaluating  the 
military  worth  and  effectiveness  of  an  NCW-enabled  force.  The  specific  type  of  tool 
within  M&S  which  will  be  utilized  for  this  effort  is  agent-based  modeling  (ABM).  The 
software  selected  for  analyzing  the  NCW  scenario  is  the  System  Effectiveness  and 
Analysis  Simulation  (SEAS).  The  SEAS  scenario  will  consist  of  generic  blue  and  red 
combat  forces,  which  will  legitimately  represent  some  NCW  capabilities  within  the 
context  of  a  mission  level  simulation  model.  As  mentioned  in  the  Research  Objective 
section  above,  a  baseline  case  of  this  scenario  will  be  run  and  compared  versus  modified 
configurations  of  the  scenario  which  removes  or  degrades  certain  NCW  capabilities. 
Based  on  the  output  from  replications  of  this  model,  various  statistical  analysis 
techniques  will  be  employed  as  tools  in  determining  the  overall  value  of  NCW  within  the 
context  of  this  thesis  research. 
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1.5  Thesis  Overview 


Following  this  Introduction  are  chapters  for  a  Literature  Review,  Methodology, 
Analysis,  and  Conclusions.  The  Literature  Review  (Chapter  2)  covers  several  definitions 
of  NCW  which  have  been  gleaned  from  foundational  works  on  the  subject,  as  well  as 
several  fundamental  definitions  laid  out  by  Joint  and  USAF  doctrinal  documents. 

Chapter  2  then  presents  a  formulated  definition  of  NCW  that  is  uniquely  crafted  for  this 
research  effort  and  concludes  with  various  possible  approaches  for  modeling  and 
measuring  NCW.  The  Methodology  chapter  (Chapter  3)  describes  the  modeling 
approach  that  will  be  used  to  represent  NCW  within  the  context  of  a  mission-level 
combat  model.  Details  and  background  of  the  SEAS  Kosovo  scenario  will  be  provided 
here.  The  Analysis  chapter  (Chapter  4)  will  provide  a  presentation  and  interpretation  of 
the  results  from  repeated  runs  of  the  combat  model  NCW  scenario  for  a  baseline  case 
versus  a  case  that  was  modified  in  order  to  determine  the  military  worth  of  applying 
NCW  within  scenario.  The  Conclusions  chapter  (Chapter  5)  provides  various  bottom- 
line  statements  derived  from  the  modeling  and  analysis  of  this  NCW  research.  Also, 
recommendations  for  further  research  of  NCW  are  offered  in  this  chapter. 
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II.  Literature  Review 


2.1  Introduction 

The  United  States  military  is  currently  undergoing  a  phase  of  revolutionary 
change  and  transition.  Paradigms  are  shifting  from  a  Cold  War,  force-on-force 
philosophy  of  warfare  to  an  asymmetric,  network  centric  approach.  The  DoD’s  Office  of 
Force  Transformation  has  issued  a  new  strategy  for  achieving  this  transition.  Vital  to  this 
transformation  effort  are  the  concepts  of  Effects  Based  Operations  (EBO)  and  NCW, 
which  will  be  covered  in  more  detail  in  section  2.2.  Having  thus  portrayed  NCW  as  a 
key  enabler  of  EBO  and  as  a  key  means  of  achieving  U.S.  military  transfonnation,  a 
thorough  exploration  of  current  NCW  definitions  and  a  fonnulated  definition  for  this 
research  effort  will  be  detailed  in  section  2.3.  Once  defined,  various  approaches  for 
modeling  NCW  will  be  described  in  section  2.4.  After  a  brief  introduction  for  this 
section,  an  exploration  of  Agent-Based  Modeling  (ABM)  as  a  possible  tool  for  modeling 
NCW  is  presented  in  section  2.4.2.  A  specific  application  of  ABM,  the  System 
Effectiveness  Analysis  Simulation  (SEAS),  is  then  described  as  prime  candidate  software 
to  model  NCW  in  section  2.4.3.  Measures  of  effectiveness  (MOEs)  for  NCW  are 
described  in  section  2.5,  followed  by  a  summary  of  this  Literature  Review  in  section  2.6. 
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2.2  Transformation,  Effects  Based  Operations  (EBO),  and  NCW 


2.2.1  Transformation  -  The  New  U.S.  Military  Strategy 
The  current  climate  of  the  United  States  military  as  a  whole  is  one  of  urgent  and 
necessary  change.  In  a  speech  at  The  Citadel  in  December  of  2001,  President  Bush 
stated: 


The  need  for  military  transformation  was  clear  before  the  conflict  in  Afghanistan, 
and  before  September  the  1 1th.  .  .  What’s  different  today  is  our  sense  of  urgency 
the  need  to  build  this  future  force  while  fighting  a  present  war.  It’s  like 
overhauling  an  engine  while  you’re  going  at  80  miles  an  hour.  Yet  we  have  no 
other  choice.  (Director,  Office  of  Force  Transformation,  2003:1) 


The  events  of  9/1 1  exposed  vulnerabilities  within  the  Department  of  Defense  that  are  no 
longer  being  ignored.  As  the  President  stated,  the  need  for  transformation  of  the  military 
was  present  prior  to  the  horrific  terror  attacks  on  the  United  States.  Unfortunately,  as  is 
the  case  with  most  human  endeavors,  proper  motivation  was  necessary  to  provide  the  fuel 
for  real  change,  which  in  this  case  is  the  full  implementation  of  military  transfonnation. 


The  current  vision  for  transformation  stated  by  the  Department  of  Defense  is  as 
follows: 


Military  transformation  will  enable  the  U.S.  Armed  Forces  to  achieve  broad  and 
sustained  competitive  advantage  in  the  21st  century.  It  comprises  those  activities 
that  anticipate  and  create  the  future  by  coevolving  concepts,  processes, 
organizations,  and  technologies  to  produce  new  sources  of  military  power.  The 
transfonnation  of  our  armed  forces  will  dramatically  increase  our  strategic  and 
operational  responsiveness,  speed,  reach,  and  effectiveness,  making  our  forces 
increasingly  precise,  lethal,  tailorable,  agile,  survivable,  and  more  easily 
sustainable.  (Director,  Office  of  Force  Transfonnation,  2003:4) 
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Essentially,  transformation  is  the  shaping  and  molding  of  our  military  force  that  seeks  to 
fully  exploit  the  advantages  we  currently  possess  and  to  protect  against  and  minimize  our 
vulnerabilities.  Transformation  is  employed  and  accomplished  through  a  combination  of 
concepts,  capabilities,  people,  and  technology.  The  overall  objective  of  these  changes  is 
to  sustain  the  U.S.  competitive  advantage  in  warfare  (Director,  Office  of  Force 
Transformation,  2003:8). 

An  essential  concept  that  drives  transformation  is  the  idea  of  forward  deterrence, 
which  is  a  stance  of  prevention  rather  than  reaction.  As  Secretary  of  Defense  Donald 
Rumsfeld  said  at  the  National  Defense  University  in  January  of  2002,  “We  must  promote 
a  more  entrepreneurial  approach  to  developing  military  capabilities,  one  that  encourages 
people,  all  people,  to  be  proactive  and  not  reactive...”  (Director,  Office  of  Force 
Transformation,  2003:29).  Although  current  U.S.  military  capabilities  are  superior  to  any 
existing  conventional  threat,  our  supremacy  will  rapidly  diminish  over  time  if  we  do  not 
continue  to  enhance  our  military  prowess  (Director,  Office  of  Force  Transformation, 
2003: 12).  There  are  several  key  components  of  transformation  that  are  geared  towards 
achieving  forward  deterrence.  Among  these  key  components  are  EBO  and  NOW.  These 
two  concepts  are  being  explored  and  refined  in  order  to  understand  how  they  complement 
one  another  in  meeting  the  needs  of  the  new  security  environment  (Smith,  2002:xxii). 

2.2.2  EBO  Fundamentals 

Unlike  network-centric  operations,  which  have  emerged  from  the  technologies 
and  thinking  of  the  Infonnation  Age,  effects-based  operations  are  not  new  (Smith, 
2002:xxiii).  Military  leaders  and  planners  have  always  tried  to  plan  and  execute  battle 


plans  and  create  battlefield  conditions  favorable  to  the  achievement  of  their  objectives 
and  policy  goals.  Rather  than  a  new  fonn  of  warfare,  EBO  is  a  way  of  thinking  or  a 
methodology  for  planning,  executing,  and  assessing  operations  designed  to  attain  specific 
effects  that  are  required  to  achieve  desired  national  security  outcomes  (Director,  Office 
of  Force  Transformation,  2003:34). 


Figure  1.  EBO's  Major  Components  (Doctrine  and  Education  Group,  2004:8) 


Figure  1  shows  EBO’s  three  major  components:  effects-based  planning, 
execution,  and  assessment  (Doctrine  and  Education  Group,  2004:8).  The  central,  cyclic 
flow  around  “ON A”  in  this  figure  represents  EBO’s  dependence  on  the  continual 
function  of  something  called  Operational  Net  Assessment  (ONA).  ONA  integrates 
people,  processes,  and  tools  that  use  multiple  information  sources  and  collaborative 
analysis  to  build  a  common,  shared,  holistic  knowledge  base  of  the  operational 
environment  (Doctrine  and  Education  Group,  2004:9).  ONA  places  primary  focus  on  the 
operational  level  and  prioritizes  the  network  as  the  key  element  of  effective  operations. 
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EBO  is  primarily  about  focusing  knowledge,  precision,  speed,  and  agility  on  the 
enemy  decision-makers  to  degrade  their  ability  to  take  coherent  action  rather  than 
conducting  combat  operations  on  more  efficient  destruction  of  the  enemy  (Director, 
Office  of  Force  Transformation,  2003:34).  As  will  be  detailed  more  fully  in  the  next 
section  concerning  NCW,  EBO  is  focused  less  on  effects  within  the  physical  domain  and 
more  so  on  effects  in  the  information  and  cognitive  domain,  with  a  special  emphasis  on 
the  cognitive,  or  decision,  arenas  of  warfare.  The  knowledge,  precision,  speed,  and 
agility  brought  about  by  network-centric  operations  provide  the  necessary  ingredients  for 
entry  into  the  realm  of  EBO  (Director,  Office  of  Force  Transformation,  2003:34). 

2.2.3  NCW -A  Key  Enabler  of  EBO 

Network-centric  warfare  is  an  emerging  theory  of  war  in  the  Information  Age.  It 
is  also  a  concept  that,  at  the  highest  level,  constitutes  the  military’s  response  to  the 
Information  Age  (Director,  Office  of  Force  Transformation,  2005:3).  In  an  increasingly 
information-driven  world,  power  is  progressively  being  drawn  from  the  sharing  of 
information,  the  degree  of  information  access,  and  speed  of  information  transmission  and 
reception.  As  an  organizing  principle,  NCW  accelerates  our  ability  to  know,  decide,  and 
act  by  linking  sensors,  communications  systems,  and  weapons  systems  in  an 
interconnected  grid  (Director,  Office  of  Force  Transformation,  2003:13).  NCW  involves 
a  modern  way  of  organizing  and  thinking  about  the  application  of  our  military  forces  as 
they  relate  to  desired  outcomes  and  therefore  is  a  key  element  of  EBO. 

A  basic  understanding  of  NCW  can  be  obtained  by  examining  the  three  domains 
of  conflict:  the  cognitive  domain,  information  domain,  and  physical  domain.  There  is 
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also  a  fourth  domain,  the  social  domain,  which  has  been  proposed  and  documented  by  the 
Office  of  Force  Transformation  as  of  January  2005.  However,  for  this  thesis  effort,  the 
focus  will  be  on  the  originally  proposed  three  domains  of  NCW.  The  three  domains 
provide  a  general  framework  for  tracing  what  actually  goes  on  in  the  stimulus  and 
response  process  inside  human  minds  and  human  organizations  (Smith,  2002:161). 

Physical  actions  often  have  a  psychological  impact,  which  is  then  translated  into  a 
decision.  The  physical  domain  spans  the  traditional  environments  of  land,  sea,  air,  and 
space  in  which  conflict  typically  occurs.  The  physical  domain  is  home  to  the  platforms 
and  communications  networks  of  a  given  military  force.  Typically,  measures  of  combat 
effectiveness  are  easiest  to  measure  in  this  domain  and  thus  it  has  traditionally  been  the 
focus  of  most  analysis  conducted  on  military  warfare.  However,  the  physical  domain 
provides  an  incomplete  picture  in  capturing  the  complex  interactions  and  outcomes  of 
real  warfare.  This  is  the  primary  reason  for  including  the  information  and  cognitive 
domains  in  the  conceptual  framework  of  NCW. 

The  information  domain  represents  the  realm  in  which  information  is  created, 
manipulated,  and  shared.  Information  traces  its  origins  to  data  collected  from  sensing 
events  in  the  physical  domain.  Comparatively,  effects  in  the  information  domain  can  be 
more  difficult  to  measure  than  those  in  the  physical  domain.  Often,  usable  measures  of 
information  are  those  pertaining  to  communication  range,  broadcast  range,  bandwidth, 
and  the  reliability  of  information  (accuracy).  This  domain  also  encompasses  all  of  the 
means  of  conveying  the  decisions,  plans,  and  orders  that  translate  a  cognitive  response 
into  physical  actions  (Smith,  2002:164-165).  Consequently,  it  is  increasingly  the 
information  domain  that  must  be  protected  and  defended  to  enable  a  force  to  generate 
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combat  power  in  the  face  of  offensive  actions  by  an  adversary  (Director,  Office  of  Force 


Transformation,  2003:33). 


The  cognitive  domain  is  the  locus  of  the  functions  of  perceiving,  making  sense  of 
a  situation,  assessing  alternatives,  and  deciding  on  a  course  of  action  (Smith,  2002:173). 
This  domain  exists  within  the  mind  of  the  warfighter.  This  is  the  realm  of  EBO 
(Director,  Office  of  Force  Transformation,  2003:33).  The  cognitive  domain  holds  the 
intangible  elements  of  knowledge,  understanding,  decision-making,  morale,  and 
leadership,  just  to  name  a  few.  Measures  for  this  domain  are  by  far  the  most  difficult  to 
assess.  Decision  analysis  methods  and  tools,  plus  an  evaluation  of  artificial  intelligence 
leadership  decisions  made  within  a  combat  model  are  possible  ways  of  capturing 


behavior  in  this  domain. 
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Figure  2.  Domains  of  Conflict  (Director,  Office  of  Force  Transformation,  2003:33) 
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Figure  2  displays  more  specifically  how  NCW  relates  to  the  three  domains. 

The  domain  intersections  represent  important,  dynamic  areas  within  which  concept- 
focused  experimentation  should  be  conducted  (Director,  Office  of  Force  Transformation, 
2003:33).  The  overlap  area  designated  Conveyed  Commander’s  Intent  (the  Shared 
Awareness  region),  where  the  information  and  cognitive  domains  intersect,  is  an 
extremely  important  realm  when  it  comes  to  the  final  outcome  of  a  given  conflict  because 
this  is  often  the  area  where  key  decisions  are  made  that  dictate  the  flow  of  warfare.  The 
intersection  of  all  three  domains  encapsulates  the  realm  in  which  NCW  exists.  NCW  is 
the  result  of  a  dynamic  interplay  of  elements  from  the  physical,  information,  and 
cognitive  domains  of  conflict. 

2.3  Defining  NCW 

2.3.1  Definitions  from  Foundational  NCW  Works 

Since  its  conception  in  the  late  1990’s,  there  has  been  a  significant  amount  of 
literature  published  on  Network  Centric  Warfare  (NCW).  Two  significant  documents, 
one  by  Alberts  (1999)  and  the  other  by  Fewell  (2003),  provide  a  baseline  from  which  to 
reference  fundamental  definitions  of  NCW.  The  Department  of  Defense  has  embraced 
the  term  and  has  included  the  concept  in  its  Joint  Vision  documents.  The  Air  Force’s 
Basic  Doctrine  also  highlights  basic  concepts  of  NCW.  From  this  plethora  of  sources, 
many  various  definitions  and  characterizations  of  NCW  will  be  distilled  and  one 
comprehensive  definition  will  be  fonned. 


13 


The  foundational  document  for  NCW  was  written  by  David  S.  Alberts,  John  J. 

Gartska,  and  Frederick  P.  Stein  in  1999.  In  their  work,  Network  Centric  Warfare, 

Developing  and  Leveraging  Information  Superiority,  Alberts  et  al.  established  a  baseline 

of  thinking  upon  which  the  structure  of  NCW  has  been  built.  Despite  the  excellent 

groundwork  that  has  been  laid,  an  exact  and  working  definition  of  NCW  is  difficult  to 

extract  from  this  foundational  text.  This  difficulty  is  largely  due  to  the  fact  that  NCW  is 

still  a  developing  idea.  As  Alberts  said  at  a  conference  in  Washington  on  March  28, 

2005,  “An  idea,  like  a  child,  takes  on  a  life  of  its  own.  It  has  parents,  it  has  supporters,  it 

has  detractors  —  all  of  which  had  a  great  influence  on  the  development  of  the  idea.  But 

ultimately  the  idea,  like  the  child,  becomes  what  it  becomes”  (Air  Force  Link,  2005).  In 

this  sense,  NCW  is  very  much  like  a  child  that  is  still  growing  and  developing  into  a 

future  form  that  is  largely  unknown  at  the  present  time. 

Despite  the  difficulties  in  finding  an  authoritative  and  accepted  definition,  certain 

key  components  of  the  current  conceptions  of  NCW  can  be  highlighted.  The  Australian 

Government’s  Defence  Science  and  Technology  Organisation  (DSTO)  compiled  an 

investigative  paper  in  2003  that  defines  NCW  in  the  following  way: 

Network-centric  warfare  is  the  conduct  of  military  operations  using  networked 
information  systems  to  generate  a  flexible  and  agile  military  force  that  acts  under 
a  common  commander’s  intent,  independent  of  the  geographic  or  organisational 
disposition  of  the  individual  elements,  and  in  which  the  focus  of  the  warfighter  is 
broadened  away  from  the  individual,  unit  or  platform  concerns  to  give  primacy  to 
the  mission  and  responsibilities  of  the  team,  task  group  or  coalition.  (Fewell  and 
Hazen,  2003:39) 

This  same  paper  identifies  four  distinct  qualities,  or  tenets,  that  are  fundamental  to  NCW: 

1)  A  robustly  networked  force  improves  information  sharing 

2)  Information  sharing  and  collaboration  enhances  the  quality  of  infonnation  and 
shared  situational  awareness 
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3)  Shared  situational  awareness  enables  collaboration  and  self-synchronization, 

and  enhances  sustainability  and  speed  of  command 

4)  These,  in  turn,  dramatically  increase  mission  effectiveness 

(Fewell  and  Hazen,  2003:2) 

The  DSTO  further  cites  NCW  as  typically  being  expressed  in  terms  of  ‘four  rights’:  the 
network  supplies  the  right  information  at  the  right  time  in  the  right  form  to  the  right 
person  (Fewell  and  Hazen,  2003:2).  The  DSTO  adds  a  fifth  ‘right’  to  this  list.  Not  only 
does  the  right  infonnation  need  to  be  available  to  the  right  person  at  the  right  time  in  the 
right  form,  but  also  it  needs  to  be  put  to  the  right  use  (Fewell  and  Hazen,  2003:34). 

Alberts  states  that  NCW  is  about  human  and  organizational  behavior  (Alberts, 
Gartska,  and  Stein,  1999:88).  This  is  an  important  point  because  it  hints  at  the 
difficulties  in  fully  modeling  NCW,  given  the  significant  human  component  and  presence 
of  complexity  due  to  human  decision  makers  and  operators.  Further,  Alberts  states  that 
NCW  is  not  narrowly  about  technology,  but  broadly  about  an  emerging  response  to  the 
Information  Age  (Alberts,  Gartska,  and  Stein,  1999:88).  In  defining  NCW,  Alberts 
cautions  that  focusing  exclusively  on  communications  networks  is  a  mistake  and  that  the 
focus  needs  rather  to  be  on  warfare  and  operations.  The  communications  networks  are 
nearly  a  means  to  an  end,  with  the  end  in  mind  being  a  more  efficient  and  effective 
conduct  of  warfare. 

2.3.2  Joint  and  USAF  Guidance  on  NCW 

The  two  core  documents  for  future  Joint  Force  operations  are  Joint  Vision  2010 
and  Joint  Vision  2020.  Joint  Vision  2010,  since  it  was  drafted  in  1997  makes  no  mention 
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of  NCW  specifically.  However,  this  document  does  comment  on  the  importance  of 

Information  Superiority,  as  shown  by  this  excerpt: 

Information  superiority  will  require  both  offensive  and  defensive  information 
warfare  (IW).  Offensive  information  warfare  will  degrade  or  exploit  an 
adversary’s  collection  or  use  of  information.  It  will  include  both  traditional 
methods,  such  as  a  precision  attack  to  destroy  an  adversary’s  command  and 
control  capability,  as  well  as  nontraditional  methods  such  as  electronic  intrusion 
into  an  information  and  control  network  to  convince,  confuse,  or  deceive  enemy 
military  decision  makers.  (DoD,  1996:  16) 

The  last  phrase  about  “nontraditional  methods  such  as  electronic  intrusion  into  an 

information  and  control  network”  signals  the  early  development  of  an  idea  that  was  to 

become  NCW  two  years  after  the  publication  of  Joint  Vision  2010,  when  Alberts 

published  his  work  in  1999. 

Joint  Vision  2020,  published  in  2000,  makes  more  certain  reference  to  the  idea  of 

network  centricity.  In  a  section  on  Information  Superiority,  Vision  2020  states: 

The  evolution  of  information  technology  will  increasingly  permit  us  to  integrate 
the  traditional  forms  of  information  operations  with  sophisticated  all-source 
intelligence,  surveillance,  and  reconnaissance  in  a  fully  synchronized  information 
campaign.  The  development  of  a  concept  labeled  the  global  information  grid  will 
provide  the  network-centric  environment  required  to  achieve  this  goal.  (DoD, 
2000:  9) 

The  mention  here  of  the  Global  Information  Grid  (GIG)  points  to  the  development  of  a 
key  component  of  NCW.  The  GIG  continues  to  be  built  and  developed.  In  early  2004, 
Mr.  Stenbit,  the  Assistant  Secretary  of  Defense  for  Networks  and  Information 
Integration,  reported  that  by  the  end  of  next  year,  DOD  plans  to  build  a  base  network 
connecting  100  locations  throughout  the  world,  involving  mostly  major  headquarters, 
intelligence  centers  and  some  support  organizations  (Stone,  2005).  Such  a  large-scale 
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communications  grid  should  serve  as  a  sufficient  hardware  foundation,  a  vast  tool  upon 

which  NCW  can  be  fully  developed,  expanded,  and  exploited. 

In  defining  the  concept  of  NCW  as  it  applies  specifically  to  the  operations  of  The 

United  States  Air  Force,  Air  Force  Basic  Doctrine  of  17  November  2003  provides  key 

insight  into  what  this  definition  might  look  like.  While  the  exact  term  “network  centric 

warfare”  is  not  present  in  this  document,  there  is  reference  to  network  warfare  and 

network  warfare  operations,  both  of  which  fall  under  the  main  category  of  information 

operations.  The  following  definition  would  seem  to  be  the  closest  match  to  a  current 

definition  of  NCW  in  the  context  of  the  USAF: 

Network  warfare  operations  are  the  integrated  planning  and  employment  of 
military  capabilities  to  achieve  desired  effects  across  the  digital  battlespace. 
Network  warfare  operations  are  conducted  in  the  information  domain,  which  is 
composed  of  hardware,  software,  data,  and  human  components.  Within  this 
domain  are  the  networks  on  which  our  information  and  infonnation  systems 
operate.  Networks  in  this  context  are  defined  as  any  collection  of  systems 
transmitting  infonnation.  This  includes  but  is  not  limited  to  radio  nets;  satellite 
links;  tactical  digital  information  links  (TADIL);  telemetry;  digital  track  files  and 
supervisory  control  and  data  acquisition  (SCAD A)  systems;  telecommunications; 
and  wireless  communications  networks  and  systems.  The  operational  elements  of 
network  warfare  operations  are  network  attack,  network  defense,  and  network 
support.  (DAF,  2003:  47) 

This  definition  provides  a  solid  baseline  for  “network  warfare”.  However,  the  constantly 
evolving  concept  of  NCW  has  come  to  entail  much  more  than  this  USAF  doctrine 
definition  captures. 

A  current  working  definition  for  NCW  being  used  by  the  (XPS)  of  Air  Combat 
Command  (ACC)  at  Langley  AFB  is  as  follows:  “Network-centric  warfare  is  the  concept 
of  linked  sensors,  communications  systems,  and  weapons  systems  in  an  interconnected 
grid  that  allows  for  a  seamless  information  flow  to  warfighters,  policy  makers,  and 
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support  personnel  (ACC/XPS,  2004: 10).  This  definition  captures  key  words  that  are 
essential  to  defining  NCW:  linked  sensors,  communications  systems,  interconnected 
grids,  weapons  systems,  seamless  information,  and  last  but  not  least,  warfighters, 
policy  makers,  and  support  personnel  -  the  humans  in  the  loop.  These  are  the  kinds  of 
tenns  that  characterize  the  essence  of  NCW. 

2.3.3  Definition  of  NCW for  this  Effort 

This  research  effort  is  primarily  focused  on  representing  an  NCW  scenario  within 
a  combat  model,  specifically  within  the  System  Effectiveness  Analysis  Simulation 
(SEAS).  With  this  specific  application  and  exercise  of  measuring  NCW  in  mind,  a 
somewhat  customized  definition  of  NCW  must  necessarily  be  formulated  to  conceptually 
match  this  application  and  provide  a  sufficient  doctrinal  baseline  to  guide  this  research. 
The  following  definition  of  NCW  has  been  formulated  for  this  effort: 

Network  Centric  Warfare  is  the  conduct  of  military  operations  through  the 
utilization  of  networked  information  systems,  which  supply  the  warfighter  with  the 
right  information  at  the  right  time  in  the  right  form  to  the  right  person  being  put  to 
the  right  use,  in  order  to  achieve  desired  effects  across  the  physical,  information, 
and  cognitive  domains  of  warfare. 

2.4  Modeling  NCW 

2.4.1  Introduction 

As  challenging  as  it  is  to  fonnulate  a  current  and  accurate  definition  of  NCW,  it  is 
perhaps  even  more  challenging  to  take  this  definition  and  then  represent  NCW  within  the 
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context  of  a  combat  model.  There  are  several  tools  and  approaches,  however,  which 
should  prove  very  useful  in  modeling  NCW. 

Agent-Based  Modeling  (ABM)  provides  an  effective  representation  of  what 
Kewley  and  Larimer  call  the  critical  gap  in  military  modeling  capabilities,  the  ability  to 
model  how  a  combat  soldier  makes  a  tactical  decision  (Kewley  and  Larimer,  2003:10). 
The  ability  to  represent  agent  decision  making  relates  well  to  modeling  NCW  because  the 
utility  and  overall  effectiveness  of  a  network  cannot  be  properly  evaluated  without  an 
accurate  representation  of  the  entities  using  the  network  and  interacting  within  the 
network.  Kewley  and  Larimer  state  that  the  increased  capability  of  network-centric 
forces,  if  it  really  exists,  is  an  emergent  property  that  cannot  be  proven  with  attrition- 
based  equations  of  combat.  Figure  3  depicts  the  progression  of  stages  that  occur  in  the 
combat  decision  making  process.  ABM  has  the  capability  to  effectively  capture  the 
cognition  and  judgment  stages  that  occur  in  between  the  data/information  levels  and  the 
final  decision  to  act.  ABM  does  this  through  a  set  of  pre-assigned  rules  given  to  agents 
within  the  model,  which  allow  the  agents  to  respond  accordingly  to  inputs  and 
conditions. 
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Figure  3.  Combat  Decision  Making  Pyramid  (Kewley  and  Larimer,  2003:10) 


SEAS  is  a  particular  type  of  agent-based  model  that  is  well  suited  for  use  as  a  tool 
in  modeling  NCW.  SEAS  is  a  model  in  the  Air  Force  Standard  Analysis  Toolkit  and  is 
widely  used  for  space  mission  utility  studies  (Walsh,  Roberts,  and  Thompson,  2005:5-6). 
SEAS  utilizes  the  fundamental  principles  of  agent  attributes  and  orders  to  model  mission 
level  combat  scenarios  and  will  serve  as  the  primary  modeling  tool  for  this  thesis 
research.  More  specific  details  about  ABM  and  SEAS  are  presented  in  the  following  two 
sections. 

2.4.2  Agent-Based  Modeling  (ABM) 

Agent-based  logic  and  programming  is  a  relatively  new  approach  to  modeling  in 
the  military  M&S  community,  tracing  its  roots  to  an  initiative  started  within  the  U.S. 
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Marine  Corps.  In  October  1995,  at  the  direction  of  the  Commanding  General  of  the 
United  States  Marine  Corps  Combat  Development  Command  in  Quantico,  two  scientists 
embarked  on  what  is  now  called  Project  Albert  (Brandstein,  Home,  and  Friman, 

2000:64).  Project  Albert  used  a  combination  of  new  models  and  tools,  multidisciplinary 
teams,  and  the  scientific  method  to  understand  how  agent-based  modeling  techniques 
could  be  correctly  applied  to  represent  a  broad  spectrum  of  military  operations.  In 
summary,  Project  Albert  was  designed  to  develop  new  tools  to  capture  emergent  behavior 
in  synthetic  environments  that  over  time  will  lead  to  more  effective  maneuver  warriors 
(Brandstein,  Home,  and  Friman,  2000:65). 

In  addition  to  Project  Albert,  another  significant  element  of  the  development  of 
ABM  was  Irreducible  Semi-Autonomous  Adaptive  Combat  (ISAAC).  Dr.  Andy 
Ilachinski  developed  the  complex  adaptive  model  to  simulate  the  interactions  between 
small  groups  of  marines  (Tighe,  1999:33).  Ilachinski  detennined  that  classical 
Lanchester-based  models  were  not  well  suited  for  modeling  the  way  in  which  the 
Marines  conducted  their  operations.  The  small,  independent,  and  well-trained  marine 
units  did  not  behave  according  to  the  mass  attrition  rates  and  large  force-on-force 
representations  of  warfare  which  Lanchester  equations  were  originally  formulated  to 
model. 

Riding  the  momentum  of  Project  Albert  and  ISAAC,  ABM  has  since  emerged  as  a 
modeling  technique  that  is  more  realistic  for  today’s  combat  scenarios  than  are  the 
classical  Lanchester-based  models.  Lanchester  equations  are  deterministic  differential 
equations.  The  unalterable  outcome  of  combat  adjudication  is  based  on  the  starting  troop 
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strengths  and  their  attrition  rates  (Tighe,  1999:28).  These  equations  provide  a  very 
simplistic  and  intuitive  framework  for  modeling  warfare.  However,  Lanchester  equations 
are  very  limited  when  it  comes  to  representing  the  complex  interactions  of  real-world 
combat  because  of  their  high  degree  of  aggregation  and  constant  attrition  rate  factors. 
Perhaps  the  greatest  strength  of  ABM  is  its  ability  to  effectively  represent  the  random  and 
unpredictable  behavior  of  entities  within  a  system,  as  well  as  the  consequent  outcomes 
resulting  from  interactions  of  such  entities.  The  effects  of  random  individual  agent 
behavior  and  of  the  resulting  interactions  of  agents  are  phenomenon  that  traditional 
Lanchester  equation-based  models  simply  cannot  capture. 

The  basic  idea  of  agent-based  modeling  is  that  autonomous  agents  are  given  a  set 
of  rules,  which  determine  how  they  will  respond  to  a  set  list  of  inputs  or  conditions 
within  the  model.  An  agent-based  model  is  one  in  which  the  connections  and 
interactions  among  the  agents  has  significant  effects,  as  compared  to  the  individual 
actions  of  any  particular  agent  (Kewley  and  Larimer,  2003: 1 1).  A  basic  summary 
definition  of  ABM  from  the  SEAS  website  is  as  follows: 

In  agent-based  modeling,  complex,  real-world  systems  are  modeled  as  collection 
of  autonomous  decision  making  entities,  called  agents.  Each  agent  individually 
assesses  its  situation  and  makes  decisions  based  upon  its  own  set  of  rules.  Agents 
may  execute  various  behaviors  appropriate  for  the  system  they  represent  -  for 
example,  sensing,  maneuvering,  or  engaging.  (SPARTA,  Inc.,  2005) 

ABM  results  in  a  realistic  simulation  of  a  system  because  it  emulates  the  manner  in 
which  the  world  really  operates  (Cares,  2002:935).  Red  and  Blue  forces  make  up  a 
dynamic,  non-linear,  complex  adaptive  system  in  which  the  overall  system  behavior 
emerges  from  the  aggregate  interactions  among  individual  agents  (Cares,  2002:936). 
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2.4.3  SEAS 


SEAS  is  a  constructive,  agent-interaction  based  simulation  designed  specifically 
for  exploratory  analysis  of  transformational,  information-driven  warfare  across  surface, 
air  and  space  domains  (SPARTA,  Inc.,  2005).  It  is  an  agent  based  combat  model 
developed  and  maintained  by  SPARTA,  Inc.  for  the  Space  and  Missile  Systems  Center 
Directorate  of  Transformation  and  Development  (SMC/TD).  SEAS  is  one  of  the  models 
in  the  Air  Force  Standard  Analysis  Toolkit.  SEAS  is  quickly  becoming  a  popularly 
utilized  software  tool  in  the  defense  M&S  community,  especially  within  the  USAF. 


SEAS  has  the  ability  to  model  the  presence  and  interaction  of  a  large  variety  of 
unique  agents  within  a  combat  mission  scenario.  Some  examples  of  the  agents  that  can 
be  represented  in  SEAS  are  tanks,  SAM  sites,  UAVs,  fighter  jets,  and  satellites.  A 
typical  mission  scenario  which  SEAS  has  the  capability  of  representing  is  shown  in 
Figure  4. 


Figure  4.  SEAS  mission  scenario  representation  (SPARTA,  Inc.,  2005:slide  2) 
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As  illustrated  in  the  above  graphic,  SEAS  can  not  only  represent  various  combat 
agents,  but  also  their  respective  sensors  and  communication  devices.  SEAS  is  built 
around  three  simple  entities:  agents,  devices  and  environments.  Essentially,  agents 
interact  through  the  use  of  devices  (weapons,  sensors,  communication)  with  each  other 
and  the  environment.  Conflict  outcomes  emerge  from  these  resulting  interactions. 

Agents  are  logical  members  acting  within  the  combat  mission  scenario.  They  can  be 
units,  such  as  a  brigade  or  multi-ship  formation  of  planes,  or  subunit  members  such  as  a 
vehicle,  individual  plane,  or  satellite.  Devices  are  entities  such  as  communications 
devices,  sensors,  and  weapons.  The  environment  is  the  battlespace,  which  consists  of 
events,  locations,  terrain,  weather,  jamming,  and  day/night  characteristics. 

A  SEAS  agent  has  the  capability  to  move  around,  sense  things,  talk  to  other 
agents,  utilize  and  acquire  resources,  and  kill  other  agents  in  an  environment.  Agents  can 
be  assigned  orders  from  superiors  and  can  also  be  given  “local  programming”  that  will 
override  the  original  orders  in  a  given  situation,  if  certain  requirements  and  conditions 
are  met.  Agents  can  also  play  various  roles  such  as  an  observer,  killer,  or  even 
leader/controller  of  other  agents.  Each  agent  with  sensing  capability  keeps  a  list  of 
targets  to  be  prepared  to  carry  out  an  order  either  to  1)  do  nothing,  2)  move  toward  them, 
3)  move  away  from  them,  4)  tell  others  about  them,  or  5)  kill  them  or  perform  some 
combination  of  the  above  (SPARTA,  Inc.,  2005  :slide  5).  Agents  and  their  respective 
interactions  follow  four  key  concepts:  the  local  target  list  (LTL),  local  orders  list  (LOL), 
target  interactions  range  (TIR),  and  broadcast  interval  (BI).  All  four  of  these  key 
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concepts  interact  with  each  specific  type  of  agent  and  the  scenario  environment  to 
produce  conflict  outcomes  within  SEAS. 


2.5  Measures  of  NCW 

There  are  several  difficulties  faced  when  trying  to  form  a  clear  definition  of  NCW 
and  formulate  an  appropriate  model  to  represent  it.  The  task  of  determining  appropriate 
and  measurable  metrics  for  NCW  also  poses  a  difficult  and  unique  challenge.  There  are  a 
wealth  of  measures  that  have  been  fonnulated  to  date  and  recorded  in  various  documents 
and  references.  For  example,  Fewell  and  Hazen  provide  a  comprehensive  list  in  the  form 
of  several  tables  which  describe  a  large  number  of  possible  NCW  metrics.  Alberts  laid  a 
basic  guideline  for  metrics,  as  shown  by  Figure  5. 


5  .Measures  of  Military  Utility 

f 

4  “  Measures  of  Exploiting  Battlespace  Knowledge 

2  —  Measures  of  Battlespace  Awareness 


r 


1  Measures  of  Infostructure  Performance 


Figure  5.  Alberts  Baseline  Metrics  (Alberts,  Gartska,  and  Stein,  1999:219) 


Infostructure  Performance,  Battlespace  Awareness,  Battlespace  Knowledge, 
Exploiting  Battlespace  Knowledge,  and  Military  Utility  are  general  categories  under 
which  more  exactly  defined  metrics  for  NCW  fall.  Fewell  and  Hazen  describe  metrics 
for  the  characteristic  ‘speed  of  command’,  force  agility  and  the  ability  to  amass  effects, 
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the  ‘degree  of  autonomy’  aspect  of  self-synchronization,  the  level  of  shared  situational 
awareness,  the  conduct  of  effects-based  operations,  reachback  operations,  infonnation 
superiority,  the  degree  of  interoperability,  and  mutual  trust.  All  total,  thirty-three 
different  metrics  falling  under  these  main  headings  are  described  in  their  document. 
However,  as  Fewell  and  Hazen  point  out,  none  of  these  metrics  serve  as  an  indicator  of 
the  level  of  network  centricity  even  though  they  do  describe  characteristics  of  net-centric 
systems  (Fewell  and  Hazen,  2003:37).  Further,  they  propose  that  the  key  characteristic 
of  network  centricity  is  the  broadening  of  warfighter  focus  away  from  the  individual,  unit 
or  platfonn  concerns  to  give  primacy  to  the  mission  and  responsibilities  of  the  team,  task 
group  or  coalition.  Quantifying  this  ‘broadening  of  focus’  is  a  difficult  problem, 
especially  when  one  tries  to  do  so  in  a  sense  that  is  independent  of  a  specific  scenario. 

Ling,  Moon,  and  Kruzins  (2005)  propose  more  quantifiable  metrics  for  measuring 
network  centric  warfare  in  the  form  of  connectivity,  reach,  richness,  and  characteristic 
tempos.  Figure  6  shows  interactions  between  the  OODA  loop  and  these  various  metrics. 
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Figure  6.  OODA  Cycle  with  Proposed  Metrics  (Ling,  Moon,  and  Kruzins,  2005:10) 


Perhaps  the  simplest  and  most  straight-forward  place  to  start  in  quantifying  and 
measuring  a  force’s  degree  of  NCW  capability  is  to  focus  on  network  transmission  delay 
time  and  the  corresponding  time  required  to  make  a  decision  to  act.  This  second  metric, 
decision  time,  may  be  more  difficult  to  track  and  measure  than  network  delay  time. 
SPARTA  proposes  the  use  of  NETE  and  SEAS  as  a  way  to  measure  network  delay  time, 
stating  that  one  way  to  use  these  tools  together  is  to  use  measures  of  performance 
(MOPs)  from  NETE  to  represent  network  delay  times  in  the  SEAS  model  where  the 
overall  campaign  is  simulated  (Walsh,  Roberts,  and  Thompson,  2005:6). 


2.6  Summary 
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The  push  for  military  transformation  received  significant  motivational  energy 
when  terrorists  invaded  our  homeland  with  a  domestic  aerial  invasion  on  September  1 1th, 
2001.  The  stark  realization  that  our  nation  was  not  supremely  safe  and  secure  elevated 
the  cause  of  military  transfonnation  and  modernization  from  an  identified  need  to  an 
urgent  and  absolute  necessity.  The  concept  of  EBO  has  been  employed,  with  NCW 
being  recruited  as  a  key  enabler  of  EBO,  to  meet  this  new  demand  for  maximized 
situational  awareness  and  decision-making  supremacy.  Having  established  NCW  as  a 
critical  area  of  military  exploration  and  application,  the  natural  follow-on  activity  of 
clearly  defining  NCW  has  presented  a  unique  and  continually  morphing  challenge. 
Several  key  documents  and  resources,  including  David  S.  Albert’s  foundational  work  on 
NCW  and  the  Joint  Vision  documents,  were  utilized  in  the  formulation  of  a  fundamental 
definition  of  NCW  for  this  research  effort.  Following  this  conceptual  framework  for 
NCW,  options  for  modeling  NCW  were  discussed.  In  particular,  agent-based  modeling 
was  described  and  proposed  as  a  legitimate  way  to  represent  the  interactions  and 
concepts  of  NCW.  A  specific  application  of  ABM,  the  simulation  software  SEAS,  was 
selected  as  the  chosen  tool  for  constructing  a  scenario  for  use  in  analyzing  the  military 
worth  of  NCW.  Finally,  several  options  for  measures  of  effectiveness  were  described 
and  a  few  key  measures  were  chosen  for  the  purposes  of  representation  and  analysis  in 
SEAS. 
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III.  Methodology 


3.1  Overview 

A  SEAS  scenario  possessing  a  relatively  high  degree  of  complexity  is  required  to 
adequately  characterize  the  key  elements  of  conducting  NCW,  namely  the  operation  and 
coordination  of  sensors,  communication  devices,  weapons  systems  and  decision-making 
entities.  An  appropriate  scenario  which  meets  these  criteria  had  already  been  created 
prior  to  this  thesis  effort  and  was  utilized  by  DeStefano  and  Zinn  for  their  collaborative 
thesis  efforts  in  2004.  The  scenario  was  written  in  SEAS  to  represent  a  mission  scenario 
typical  of  the  Kosovo  conflict  during  1999. 

The  following  sections  provide  a  description  of  the  Kosovo  scenario  background, 
warfile,  verification  and  validation  (V&V),  and  NCW  features.  Then,  the  selected 
measures  that  will  be  extracted  from  the  Kosovo  scenario  and  analyzed  for  their  military 
worth  within  the  context  of  NCW  are  described.  Next,  the  analysis  approach  describing 
the  specific  procedure  and  statistical  tools  are  covered.  Finally,  this  Methodology 
chapter  concludes  with  a  brief  summary  of  all  the  topics  covered  and  important  points  to 
keep  in  mind  before  proceeding  to  the  next  chapter,  Analysis. 

3.2  SEAS  Kosovo  Scenario 

3.2.1  Background 

The  Space  and  Missile  Center  Transformation  Directorate  (SMC/TD)  has  created 
a  warfile  in  SEAS  to  represent  a  typical  mission  in  the  Kosovo  war  (DeStefano,  2004:3- 
3).  The  SEAS  warfile  was  created  for  the  Air  Force  by  the  MITRE  Corporation  in 
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Hampton,  VA  (DeStefano,  2004:3-1).  The  scenario  consists  of  a  Blue  United  States  Air 
Forces  in  Europe  (USAFE)  force,  a  Red  Serbian  force,  and  a  Brown  Kosovar  force  of 
militia  and  civilians,  all  programmed  to  operate  and  interact  within  the  context  of  typical 
operations  in  the  Kosovo  conflict  during  1999.  It  essentially  models  Red  forces 
conducting  “ethnic  cleansing”  operations  against  the  Brown  civilians  (Zinn,  2004:48). 
Blue  force’s  objective  is  to  stop  the  Red  force  from  killing  the  Brown  force.  Blue 
achieves  this  objective  by  attacking  the  Red  force  and  by  attempting  to  contain  their 
military  operations  and  movements. 

DeStefano  utilized  the  Kosovo  scenario  as  an  architectural  data  product  to 
represent  the  Time  Critical  Targeting  (TCT)  activities  of  the  Air  Operations  Center 
(AOC)  (DeStefano,  2004:iv).  DeStefano  made  needed  additions  and  adjustments  to  the 
original  Kosovo  scenario  delivered  by  SMC/TD  to  fit  his  research  and  analysis  needs  as 
he  sought  to  demonstrate  the  significance  of  Time  Critical  Targeting  (TCT)  activities  of 
the  AOC.  The  version  used  by  DeStefano  and  Zinn  for  their  thesis  efforts  is  the  same 
version  of  the  Kosovo  scenario  that  will  be  utilized  in  modeling  NCW  for  this  research 
effort. 

3.2.2  Kosovo  Scenario  Warfile 

The  programming  code  used  within  the  SEAS  interface  is  called  Tactical 
Programming  Language  (TPL).  Multiple  lines  of  TPL  compose  a  file  designated  as  the 
“warfile”,  which  contains  all  the  necessary  infonnation  concerning  locations,  agents, 
their  sensors,  weapons,  and  communication  capabilities,  as  well  as  the  orders  followed  by 
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each  agent.  Figure  7  shows  an  example  of  SEAS  TPL  from  the  Kosovo  warfile  which 


gives  agent  attributes  and  order  for  the  Blue  SOFReconSqdEast  unit. 


1171 

Unit  "SOF  ReconSqdEast" 

1172 

ICON 

"icons\beyesico" 

1173 

4 

1174 

2 

117S 

9  ! !  Each  unit 

is  a  nine  person  team 

117€ 

Mass 

1000 

1177 

Altitude 

0 

1178 

Dig  Start 

1 

1179 

Dig  Done 

5 

1180 

Dig  Pd 

0.01 

1181 

Dig  Pk 

0.5 

1182 

Deploy  Delay 

0 

1183 

Promote  Limit 

4 

1184 

Fusion  Type 

0 

118S 

1 

1188 

100 

1187 

Comm 

"Sat  TacR" 

These  are  all  comm  devices  used  by  unit 

1188 

Comm 

"ElmtSAT  TacR" 

1189 

Comm 

"SOF  Ord" 

1190 

Comm 

"SOF  Sat  Ph  R" 

1191 

Comm 

"JSTARS  TacR" 

1192 

Comm 

"Gunship  TacR" 

1193 

Sensor 

"Distress  Detector" 

1194 

Vehicle 

"SOF  Recor.Sqd  Mem"  9 

119S0 

Orders 

1196 

Declare  Global  ReconLocs [5] ,  5CFSqioc[5], 

Gshiponmission [5] , 

1197 

Declare  Global  SOF_SAM_tgts [6] ,  SOF_ Armor 

_tgts[2],  SOF_Kos_allies [2] 

1198 

Declare  Local  ms 

*  0,  lsf  mt  *  0,  It,  km 

«  0,  kos 

1199 

Declare  Local  lastExcursion,  scoutPoint 

1200 

Declare  Local  i  - 

0,  MySpeed,  ImtTotalMass,  DnitFireThreshold-10 

1201 

SOF  SAM  tgts [0]  = 

"RedSA3RadarVar." 

! !  These  arrays  set  target  priorities 

1202 

SOF  SAM  tgts [1]  = 

"RedSA61RadarVar." 

1203 

SOF  SAM  tgts [2]  = 

"RedSA62RadarVan" 

1204 

SOF  SAM  tgts [3]  = 

"SA3Tel" 

1205 

SOF  SAM  tgts [4]  = 

"SA61Tel" 

1205 

SOF  SAM  tgts [5]  = 

"SA62Tel" 

1207 

SOF  Armor  tgts[0] 

-  "TeO" 

1208 

SOF  Armor  tgts[l] 

-  "SUV" 

1209 

SOF  Kos  allies [0] 

=  "KosVillagel  unit" 

1210 

SOF  Kos  allies [1] 

=  "KosVillage2  unit" 

Figure  7.  Example  of  TPL  Code  from  the  Kosovo  Warfile 


Each  line  of  code  is  numbered  on  the  far  left  margin.  All  of  the  unit  attributes  for 
SOF  ReconSqdEast  are  listed  in  this  block  of  TPL.  Below  the  attributes  is  a  list  of  the 
various  communication  devices  utilized  by  the  SOF  agents.  Also  shown  in  Figure  7  are 
orders  which  each  agent  will  follow  as  they  interact  in  the  scenario.  In  this  case,  the  SOF 
agents  are  assigned  a  priority  list  for  target  sighting  reporting  purposes.  Comments  in 
TPL  are  preceded  by  two  exclamation  marks  and  given  a  light  blue  color  in  the  warfile. 


31 


A  typical  SEAS  warfile  is  structured  in  sequential  blocks  that  designate  the 


timing,  location,  and  force  composition  of  the  scenario.  The  Kosovo  scenario  warfile 
follows  this  same  general  format.  The  first  lines  of  TPL  state  that  the  scenario  takes  place 
well  in  the  future,  on  August  2nd,  2016.  The  scenario  date  is,  for  all  practical  purposes, 
arbitrary.  It  simply  provides  a  timeline  reference  from  which  to  track  the  flow  of  combat 
activities.  This  TPL  for  event  timing  in  the  Kosovo  warfile  allows  for  a  possible  20-day 
scenario  that  will  end  on  August  22nd,  2016.  However,  as  noted  by  DeStefano  and 
confirmed  by  runs  for  this  thesis  effort,  no  significant  activity  occurred  after  6000 
minutes  (100  hours  or  4.17  days)  of  simulation  time,  and  no  event  based  criteria  to  stop 
the  simulation  was  uncovered  (e.g.  all  Serbian  forces  are  killed  or  withdrew)  (DeStefano, 
2004:4-2). 


Time  (hours) 


Figure  8.  Sensor  Detection  Activity  for  Key  Kosovo  Scenario  Agents 

Figure  8  confirms  the  drop-off  in  activity  as  measured  by  activity  of  sensors  for 
several  key  agents  after  100  hours.  Although  the  Global  Hawk  exhibits  a  few  detections 
for  hours  108  and  109,  all  activity  has  essentially  ceased  for  all  other  major  players  after 
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100  hours  and  therefore  this  run  time  will  also  be  used  as  the  run  time  for  multiple 
simulation  replications. 

Location  information  follows  the  event  timing  block  of  the  warfile.  A  graphical 
depiction  showing  several  of  the  key  locations  for  the  scenario  is  illustrated  in  Figure  9. 
These  location  lines  of  code  specify  key  locations  for  the  Kosovo  scenario,  all  of  which 
are  assigned  a  name  (e.g.  the  point  for  Aviano  AFB  in  the  figure’s  upper  left-hand 
comer)  and  are  coded  in  the  warfile  according  to  their  coordinates  of  latitude  and 
longitude. 


Figure  9.  Kosovo  Scenario  Locations 

This  figure  shows  several  Tactical  Area  of  Operations  (TAO)  areas,  all  of  which 
are  shown  as  irregular  shapes  bounded  with  black  lines.  The  largest  TAO, 
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BalkanWxTAO,  represents  a  region  of  weather  whose  attributes,  primarily  altitude  range 


and  intensity  factor,  degrade  communication  signals’  transmission/reception  and  sensor 


performance  occurring  in  the  areas  bounded  by  the  TAO.  Another  significant  TAO, 


KosovoTAO,  lies  within  the  BalkanWxTAO.  Also  shown  in  this  figure  are  the  GH  Orbit, 


Predator  Orbit,  Gunship  Orbit,  JSTARS  Orbit,  and  SOF  Patrol  TAOs  which  specify 
aircraft  orbits  and  troop  patrol  areas,  respectively. 


Weather 

TAOs 


Weather  affects 
Platform  speed 
Sensor  Pd 
Weapon  Pk 
Comm  Reliability 


Multiple  weather  or  terrain  regions 
can  be  defined  and  assigned  to 
the  same  entity  (unit,  platform, 
sensor,  weapon  or  comm) 

Multiple  weather  or  terrain  TAOs 
can  overlap. 


Terrain 


Terrain  affects 
Platform  speed 
Sensor  range 
Weapon  range 
Comm  range 


Figure  10.  Weather  and  Terrain  Effects  in  SEAS  (SEAS  Training  CD  Slides,  2005:  slide 

274) 

Figure  10  illustrates  the  specific  attributes  within  SEAS  that  are  affected  by 
weather  and  terrain  TAO  areas.  Weather  is  listed  as  affecting  platform  speed,  sensor 
probability  of  detection,  weapon  probability  of  kill,  and  communications  reliability. 
Terrain  is  listed  as  affecting  platfonn  speed,  sensor  range,  weapon  range,  and 
communications  range.  It  is  important  to  keep  in  mind  that  the  degradation  effects 
implemented  in  the  Kosovo  scenario  are  being  utilized  as  generic  ways  to  degrade 
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network  performance  on  a  large-scale  ( BalkanWxTAO )  and  more  local  scale 
(. KosovoTAO ),  both  of  which  affect  unique  aspects  of  performance.  The  KosovoTAO 
draws  the  boundary  for  a  terrain  region  whose  degradation  factor  degrades  the  ability  of 
the  Blue  Force’s  UAV  to  see  targets  and  therefore  makes  the  simulation  of  the  UAV 
patrolling  the  area  more  realistic.  In  other  words,  agents  will  occasionally  be  hidden 
from  the  UAV’s  view  because  the  terrain  factor  (which  ranges  from  0  to  1  in  SEAS  and 
is  set  at  0.8  for  the  KosovoTAO)  is  applied  to  all  sensing  operations  within  that  TAO  and 
will  only  allow  a  percentage  of  line  of  sight  detections  to  occur.  For  instance,  within  the 


KosovoTAO,  only  eighty  percent  of  the  target  sightings  in  that  region  will  be  officially 
recorded  as  a  clean  detection. 
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Figure  11.  Kosovo  Scenario  Weather  and  Terrain  Blocks  TPL 


Figure  1 1  shows  the  TPL  for  the  Balkan  weather  block  and  Kosovo  terrain  block. 


The  KosovoTAO  terrain  factor  of  0.8  can  be  seen  here,  as  well  as  the  Balkan  WxTAO 
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attributes  of  altitude  range  (10  to  15  kilometers)  and  degradation  factor  of  zero.  The  zero 
degradation  factor  for  BalkanWxTAO  means  that  no  communication  or  image  detection 
can  be  accomplished  if  it  has  to  pass  through  this  region.  For  instance,  if  one  of  the 
satellites  in  the  scenario  searches  for  targets  in  the  BalkanWxTAO,  it  will  not  detect 


anything  and  also  will  not  be  able  to  broadcast  any  information  into  that  region. 
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Figure  12.  Blue  Force  Structure 


Now  that  the  timing,  location,  TAO,  weather,  and  terrain  blocks  have  been 
covered,  the  TPL  sections  for  the  forces,  units,  and  vehicle  hierarchy  of  the  scenario  must 
be  described.  As  has  been  previously  mentioned,  there  are  three  forces  in  the  Kosovo 
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scenario:  a  USAFE  force,  Serbian  force,  and  Kosovar  force.  Figure  12  gives  a  graphical 
depiction  and  breakdown  of  the  Blue  USAFE  force. 

As  can  be  seen  from  this  figure,  the  Blue  force  has  a  considerable  number  of  units 
and  vehicles,  especially  in  relation  to  the  Red  and  Brown  forces,  which  are  depicted  in 
Figures  13  and  14.  All  units  for  the  Blue  force  fall  under  and  are  owned  by  the  USAF 
Combined  Aerospace  Operations  Center  (CAOC),  which  is  referred  to  as  the  “parent 
unit”  for  the  Blue  force.  The  significance  of  the  parent  unit  is  that  a  parent’s  orders  take 
precedence  over  any  orders  that  each  individual  “child  unit”  (units  that  are  subordinate  to 
the  parent)  may  have  within  their  own  code  block.  The  Blue  Force  Structure  illustration 
depicts  the  typical  force  breakdown  within  SEAS,  in  which  units  are  composed  of 
vehicles  (e.g.  the  F15_SEADSqdn  is  composed  of  multiple  F-15s),  each  having  the 
potential  of  owning  sensors,  communication  devices,  and  weapons.  For  example,  the 
Special  Operations  Forces  (SOF)  units  of  East  and  West  (West  unit  breakdown  is  not 
shown  in  Figure  12  since  its  composition  is  identical  to  the  East  unit)  both  own  the 
communication  device  SOF  Ord,  the  sensor  SOF  scope,  and  the  weapon  M4_Carbine. 
The  numbers  in  parenthesis  following  any  name  in  the  hierarchy  indicates  the  quantity  of 
a  particular  unit  or  vehicle  within  the  Kosovo  scenario.  For  instance,  the  Blue  Force  has 
two  SOF  ReconSqnEast  units,  nine  SOF _ReconSqd_Mem  vehicles,  and  the  F-15s  each 
have  two  JSOW  and  two  HARM  weapons.  While  the  Blue  force  is  quite  capable  on  the 
ground  with  the  SOF  units,  the  major  emphasis  of  the  force  is  on  air  assets  and  the 
application  of  air  power. 
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The  Red  Serbian  force,  shown  in  Figure  13,  is  much  simpler  in  comparison  to  the 
Blue  force.  The  Serbian  force  is  not  centralized  as  is  the  blue  force  possessing  the 
CAOC  unit  agent,  which  owns  all  other  blue  agents.  The  Red  force  consists  solely  of 
ground  assets  of  the  Serbian  Army.  Serbian  unit  agents  include  air  defenses,  ground 
targets,  and  three  army  divisions  (DeStefano,  2004:3-5). 


Figure  13.  Serbian  Force  Structure  (DeStefano,  2004:3-6) 


The  Serbian  surface-to-air  missile  capabilities  present  the  greatest  threat  to  the 
Blue  force  in  terms  of  attrition,  based  on  initial  experimental  runs  of  the  scenario. 
However,  since  the  goal  of  the  Blue  force  in  the  scenario  is  to  minimize  the  impact  of 
Serbian  Army  operations  on  the  Kosovars,  ultimately  the  three  Serbian  armor  units  are 
the  most  threatening  members  of  the  Red  force  in  terms  of  Blue  achieving  its  objective. 
Orders  are  passed  from  the  five  main  Serbian  unit  agents  to  their  subordinate  agents,  but 
there  is  not  the  degree  of  coordination  of  the  Blue  force  since  these  five  units  essentially 
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act  autonomously.  This  is  a  fairly  obvious,  and  yet  true  to  life,  weakness  for  the  Serbian 


force.  The  Serbian  force  behaves  according  to  a  realistic  concept  of  operations.  For 


instance,  the  surface  to  air  radar  vans  are  given  orders  to  hide  when  information  is  passed 


that  an  F-15  is  near,  or  to  hide  and  move  after  firing  a  missile  (DeStefano,  2004:3-5). 


0  CD 


Figure  14.  Kosovar  Force  Structure  (DeStefano,  2004:3-7) 


The  Brown  Kosovar  force  is  similar  to  the  Red  Serbian  force  in  the  sense  that 
there  is  no  centralized  command  structure,  as  seen  in  Figure  14.  The  Kosovars  force 
consists  of  fanners,  refugees,  villagers,  or  militia  members.  The  militia  members  are  the 
only  armed  agents  of  the  Brown  force  and  they  are  enemies  with  the  Serbian  force,  but 
are  neutral  in  relation  to  the  Blue  force.  The  Kosovar  agents  have  extremely  rudimentary 
sensing  and  transmitting  capabilities  such  as  unaided  human  eyes,  cell  phones,  and  even 
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bells,  all  of  which  are  coded  in  the  warfile  as  devices  whose  attributes  have  been  assigned 
to  match  the  low  strength  and  low  range  of  these  types  of  sensors  and  communication 
devices. 

Instead  of  the  Kosovars  being  placed  in  aggregated  masses  at  certain  locations 
they  can  be  modeled  as  agents  who  can  pass  along  infonnation  to  the  U.S.  forces  and 
hide  from  the  enemy  (DeStefano,  2004:3-5).  In  this  sense,  the  Kosovars  can  be  viewed 
as  allies  to  the  Blue  force.  However,  since  they  are  only  able  to  offer  limited  combat 
support,  they  would  more  accurately  be  labeled  as  a  neutral  force  in  this  scenario. 

3.2.3  Warfile  Verification  and  Validation 

DeStefano  describes  various  verification  and  validation  (V&V)  activities  applied 
to  the  Kosovo  scenario  warfile  in  his  thesis  effort.  He  states  that  some  of  the  standard 
methods  employed  in  the  V&V  process  for  his  effort  were  a  structured  walk-through  of 
the  code,  consultation  with  experts,  viewing  the  animation,  and  looking  for  reasonable 
output  (DeStefano,  2004:3-24).  Every  time  agent  orders  changed,  DeStefano  perfonned 
a  structured  walk-through  of  the  warfile  code  and  utilized  the  SEAS  details  and  debug 
window  to  ensure  that  global  and  local  variables  were  appropriately  updated  so  that  agent 
orders  were  correct  and  current.  Further,  DeStefano  consulted  with  experts  at  SMC  (a 
primary  user  of  SEAS),  Sparta  Inc.  (model  managers),  and  RAND  (analysts)  throughout 
his  use  and  modification  of  the  warfile  (DeStefano,  2004:3-24).  SEAS  animation  proved 
to  play  a  key  role  in  DeStefano’s  V&V  process.  For  instance,  movement  of  the  global 
hawk  away  from  its  TAO  to  investigate  a  potential  target  was  confirmed  by  viewing 
SEAS  animation  of  the  Kosovo  warfile. 
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A  few  additional  investigations  were  perfonned  during  this  thesis  research  to 
further  verify  and  validate  the  Kosovo  SEAS  model.  For  instance,  through  initial 
exploratory  checks  of  the  Kosovo  scenario  TPL  and  SEAS  animation,  it  was  observed 
that  one  of  the  scenario’s  three  satellites,  Elint_SAT,  held  an  extremely  high  altitude 
orbit.  The  orbit  was  so  high  relative  to  the  other  two  satellites  that  it  seemed  at  first  to  be 
a  programming  error.  Figure  15  shows  a  screen  capture  of  this  satellite’s  location,  as 
well  as  the  location  of  the  other  two  USAFE  satellites. 


The  lines  and  circles  emanating  from  each  satellite  show  the  sensor  field  of  regard 
for  each  respective  satellite  relative  to  the  earth.  Upon  further  investigation  of  the  warfile 
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orbit  information  (which  is  contained  in  a  *.sat  file  that  is  called  out  within  the  Kosovo 
scenario  warfile),  it  was  discovered  that  the  EIint_SAT  is  a  geosynchronous  satellite.  For 
a  satellite's  orbit  period  to  be  one  sidereal  day  (the  time  it  takes  the  Earth  to  rotate  360 
degrees,  which  is  equal  to  23  hours,  56  minutes  and  4  seconds),  it  must  be  approximately 
35,786  kilometers  (19,323  nautical  miles  or  22,241  statute  miles)  above  the  earth's 
surface  (NASA  Liftoff  Home,  1995).  Through  utilization  of  the  kilometer  scale  for 
screen  width  given  by  SEAS  in  its  graphics  display  (shown  to  be  125,802.699  kilometers 
in  Figure  15,  which  is  not  to  scale  for  this  illustration  due  to  image  cropping),  it  was 
concluded  that  Elint_SAT  is  indeed  approximately  35,786  kilometers  above  the  earth’s 
surface.  Therefore,  the  Elint_SAT  altitude  in  the  scenario  matches  the  real-world  altitude 
of  a  geosynchronous  satellite.  Also,  SAT1  and  SAT2  occupy  orbits  that  allow  them  to 
sweep  the  Kosovo  area  once  every  12  hours.  This  verification  confirms  that  the  satellite 
orbits  are  realistic  and  contributes  to  an  increased  confidence  level  that  the  Kosovo 
scenario  as  a  whole  is  written  and  composed  correctly.  Similar  checks  for  scenario 
accuracy  were  performed  for  various  other  platforms  and  agents  throughout  the  Kosovo 
warfile  in  order  to  verify  that  that  the  scenario  was  properly  coded. 

Validation  of  results  from  SEAS  was  perfonned  primarily  as  face  validation 
throughout  the  analysis  process.  This  face  validation  consisted  of  common  sense  checks 
of  the  output  values  for  detection  distances,  times  of  detections,  communication  channel 
activity,  and  kill  numbers. 
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3.2.4  Kosovo  Scenario  NCW  Features 


There  are  several  key  elements  of  the  Kosovo  scenario  that  allow  it  to  be  used  as 
a  scenario  which  legitimately  represents  and  applies  the  concepts  of  NCW.  Drawing 
from  the  various  NCW  definitions  covered  in  the  Literature  Review,  the  concept  of 
linked  sensors  was  highlighted  in  the  definition  of  NCW  used  by  Air  Combat 
Command’s  Future  Plans  (ACC/XPS)  division.  A  count  of  sensors  in  the  Kosovo  warfile 
shows  that  20  total  sensors  are  used  in  the  scenario:  13  sensors  belong  to  the  Blue 
USAFE  force,  four  sensors  belong  to  the  Red  Serbian  force,  and  three  sensors  belong  to 
the  Brown  Kosovar  force.  Some  of  these  sensors  are  shared,  such  as  the 
BluAir2GndRadar  and  AC_Elint  used  by  both  the  F-15s  and  F-16s.  ACC/XPS  also 
highlighted  linked  communications  as  another  key  component  of  NCW.  The  Kosovo 
scenario  holds  23  total  communication  channels:  17  channels  belong  to  the  Blue  USAFE 
force,  three  channels  belong  to  the  Red  Serbian  force,  and  three  channels  belong  to  the 
Brown  Kosovar  force.  Many  of  these  communication  channels,  especially  on  the  Blue 
force  side,  are  shared  between  several  different  units  and  vehicles.  The  linked  sensors 
and  communications  aspects  of  NCW  are  definitely  captured  in  the  Kosovo  scenario. 

This  interconnected  grid  of  sensing  and  communication  devices  allows  for  the  operation 
of  linked  weapons  systems  and  creates  shared  situational  awareness  in  the  scenario, 
especially  among  the  Blue  USAFE  force  units  and  vehicles. 

3.3  Selected  Measures 

Based  on  the  outputs  available  from  SEAS  and  the  analysis  options  provided  by 
the  SEAS  Post  Processor  (an  Excel-based  analysis  tool),  the  focus  for  selected  measures 
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in  this  research  has  been  placed  primarily  on  the  physical  and  infonnation  domains  of 
NCW.  For  the  physical  domain,  the  most  appropriate  measure  seems  to  be  sensor 
detection  distance.  The  SEAS  Post  Processor  provides  extensive  capability  for  the 
filtering,  graphing,  and  raw  data  analysis  of  detection  distances  for  each  sensor  active  in 
the  scenario.  The  average  detection  distances  for  key  platforms  will  be  analyzed  for 
trends  in  performance  over  the  four  cases  of  the  Kosovo  scenario.  Also,  the  average 
number  of  detections  per  replication  will  be  looked  at  for  these  same  key  platforms. 

For  the  information  domain,  the  load  on  the  communications  network  for  various 
key  channels  in  the  Kosovo  scenario  will  be  measured.  The  SEAS  Post  Processor  will 
also  be  employed  for  this  analysis,  as  well  as  use  of  Excel  to  directly  manipulate  and 
filter  the  raw  data  of  communication  outputs  from  SEAS.  SEAS  keeps  track  of  three 
communications  metrics:  the  number  of  messages  added,  the  number  of  messages 
currently  on,  and  the  number  of  messages  removed  for  each  communications  channel 
over  each  one -minute  time  step  of  the  simulation.  The  data  that  tracks  this  running  tally 
of  communications  channel  loading  will  be  utilized  to  analyze  performance  of  the  Blue 
Force’s  communications  channels  in  order  to  determine  the  effects  of  applying  various 
degradation  levels  in  the  Kosovo  region. 

Even  though  no  direct  measure  for  the  cognitive  domain  will  be  extracted  from 
the  Kosovo  scenario  for  this  effort,  an  indirect  measure  of  the  cognitive  domain  will  be 
analyzed.  The  chosen  measure  to  gauge  the  quality  and  success  of  decisions  made  by 
agents  in  the  scenario  is  the  killer  and  victim  data  tracked  by  SEAS  as  a  standard  output. 
The  Killer  Victim  Scoreboard  (KVS)  is  a  useful  tool  within  the  SEAS  Post  Processor  for 
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filtering,  analyzing,  and  presenting  infonnation  pertaining  to  the  number  of  kills 
throughout  the  scenario,  as  well  as  the  identity  of  the  killer  and  victim  and  the  timing 
during  which  the  kill  occurs.  KVS  information  will  be  compared  for  the  four  cases  in 
order  to  detennine  the  ultimate  effect  of  degrading  the  performance  of  sensors  and 
communications  equipment.  The  analysis  will  focus  on  comparing  the  number  of  Red 
Force  Serbian  agents  killed  by  Blue  Force  USAFE  agents,  the  number  of  Blue  killed  by 
Red,  and  the  number  of  Brown  Force  Kosovars  killed. 

3.4  Analysis  Approach 

Multiple  replications  of  the  Kosovo  scenario  will  be  run  in  a  configuration  that  is 
free  of  weather  and  terrain  effects.  The  measures  described  in  Section  3.3  will  be 
collected  from  these  multiple  runs  of  the  scenario  and  analyzed  to  find  the  mean  and 
standard  deviation  values  and  confidence  intervals  will  be  constructed  for  these  outputs. 
Next,  the  scenario  will  be  run  multiple  times  in  a  configuration  where  weather  and  terrain 
TAO  effects  are  applied  separately  to  degrade  the  sensing  and  communication  operations, 
respectively.  The  resulting  average  sensor  distances  for  all  sensors  detecting  enemies  in 
the  scenario  will  be  analyzed  for  both  cases.  Then,  multiple  runs  applying  both  weather 
and  terrain  effects  will  be  performed.  The  resulting  average  detection  distances  from  the 
full  weather  and  terrain  effects  scenario  will  then  be  analyzed  in  the  same  manner  as  the 
cases  applying  weather  effects  only  and  terrain  effects  only.  The  resulting  average 
detection  distances  for  all  sensing  agents  will  be  compared  to  base  case  outputs.  Output 
analysis  in  the  form  of  a  two-sided  t-test  will  be  perfonned  to  detennine  whether  the 
differences  between  the  three  configurations’  outputs  and  the  base  case  outputs  are 
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statistically  significant.  From  this  comparison  and  analysis,  various  insights  and 
conclusions  will  be  drawn  concerning  the  results  and  performance  of  all  three  forces  in 
the  Kosovo  scenario,  all  determined  and  presented  in  light  of  NCW  principles. 

3.5  Summary 

This  section  has  described  the  background,  warfile,  verification  and  validation, 
and  NCW  features  of  the  Kosovo  scenario  SEAS  warfile.  The  warfile  was  originally 
written  by  SMC/TD  and  used  by  DeStefano  and  Zinn  for  their  theses.  Essentially,  the 
Kosovo  scenario  depicts  an  ethnic  cleansing  operation  in  which  the  role  of  the  Blue 
USAFE  force  is  to  stop  the  Red  Serbian  force  from  killing  the  Brown  Kosovar  force. 
Warfile  TPL  code  analysis,  expert  consultation,  simulation  animation  checks,  and 
scenario  output  analysis  were  used  in  the  V&V  process  for  the  Kosovo  warfile.  These 
V&V  activities  were  conducted  both  by  DeStefano  for  his  thesis  effort  and  for  this 
current  effort  to  model  NCW.  Key  features  of  NCW  were  cited  as  being  present  in  the 
Kosovo  scenario.  These  NCW  features  primarily  relate  to  the  high  degree  of  linked 
sensors,  communication,  and  weapons  systems  contained  in  the  scenario.  SEAS 
measures  for  the  physical  and  information  domain  were  chosen.  For  the  physical 
domain,  target  detection  distance  is  the  selected  measure  to  be  extracted  from  multiple 
simulation  runs.  The  average  detection  distances  for  all  sensing  vehicles  and  agents  in 
the  scenario  will  be  analyzed  using  the  SEAS  Post  Processor.  For  the  information 
domain,  communications  channels  loading  and  activity  will  be  analyzed.  Both  the 
number  of  messages  and  timing  of  these  messages  throughout  the  scenario  will  be 
analyzed  and  comparisons  made  between  the  four  scenario  cases.  These  selected 
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measures  will  be  the  focus  for  analysis  of  outputs  taken  from  multiple  simulation  runs. 
The  analysis  will  determine  the  average  output  values  from  the  base  case  scenario,  as 
well  as  for  three  cases  in  which  weather  only,  terrain  only,  and  a  combination  of  weather 
and  terrain  effects  will  be  added  into  the  scenario.  The  next  chapter,  Analysis,  presents 
the  outputs  and  statistical  analysis  resulting  from  the  accomplishment  of  multiple 
simulation  replications. 
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IV.  Analysis 


4.1  Overview 

This  chapter  provides  statistical  analysis  and  results  as  well  as  a  description  of  the 
process  involved  in  determining  what  Kosovo  scenario  platforms  and  outputs  are  worth 
focusing  on  for  the  sake  of  measuring  NCW.  The  chapter  begins  by  laying  out 
preliminary  analysis  conducted  to  detennine  an  appropriate  number  of  simulation  runs. 
Next,  there  is  a  section  about  detection  distance  analysis  pertaining  to  the  physical 
domain  of  NCW,  followed  by  an  investigation  and  analysis  of  communication  channel 
loading  for  the  infonnation  domain.  The  chapter  then  concludes  with  a  section  covering 
the  analysis  of  kill  numbers,  which  serve  as  a  final  measure  of  agent  decision  output  for 
the  cognitive  domain,  followed  by  a  brief  chapter  summary. 

4.2  Selecting  the  Number  of  Simulation  Replications 

A  preliminary  task  of  simulation  analysis  is  to  select  the  number  of  replications  to 
run  in  SEAS  in  order  to  obtain  output  data  sets  which  have  desirable  statistical  properties. 
Tentatively,  100  runs  were  chosen  as  the  target  number  of  replications.  However,  due  to 
the  extensive  time  required  to  run  the  Kosovo  scenario  100  times  (one  hour  per  100  runs, 
times  four  for  each  case)  and  the  considerable  file  size  of  output  data  from  initial  checks 
of  running  the  scenario  ten,  twenty,  and  thirty  times  (sensor  output  data  files  for  thirty 
runs  were  in  the  350-450  MB  range),  it  was  discovered  that  working  with  100  runs  was 
not  a  practical  approach.  To  strike  a  balance  between  obtaining  a  sufficient  amount  of 
data  to  ensure  the  ability  to  make  legitimate  statistical  inferences,  while  at  the  same  time 
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keeping  the  time  required  to  perform  the  simulation  replications  and  output  file  sizes 


within  reasonable  limits,  thirty  simulation  replications  was  chosen  as  the  new  target. 


A  check  for  normality  was  performed  for  outputs  of  average  detection  data  for 
different  sensors  at  the  levels  of  ten,  twenty,  and  thirty  replications.  Output  data  from  the 
JSTARS  was  selected  as  the  focus  for  this  normality  check  because  preliminary  analysis 
of  model  outputs  showed  that  the  JSTARS  agent  provided  the  highest  number  of  detection 
samples  over  each  replication  and  therefore  data  from  this  platfonn  seemed  to  provide  a 
fair  representation  of  the  overall  distribution  of  data  for  platforms  in  the  scenario  as  a 
whole. 


Figure  16.  Check  for  Normality  -  Plots  from  Arena  Input  Analyzer 


Figure  16  shows  the  increasing  progression  of  data  normality  from  analysis  of  the 
average  detection  distance  output  for  the  JSTARS  resulting  from  ten,  twenty,  and  thirty 
replications.  The  square  error  for  a  normal  distribution  fit  decreases  from  0.03  to  0.007 
as  the  number  of  replications  is  increased  from  ten  to  thirty.  These  statistical  distribution 
plots  offer  support  for  the  assumption  that  thirty  simulation  runs  is  a  sufficient  number  to 
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obtain  approximately  normally  distributed  output  data  from  multiple  runs  of  the  Kosovo 
scenario. 

4.3  Physical  Domain  Analysis 

4.3.1  Single  Run  Analysis  -  Targets  and  Sensors  of  Interest 

Preliminary  analysis  of  sensor  detection  distances  for  the  physical  domain  of 
NCW  began  with  determining  which  sensors  were  programmed  in  the  Kosovo  warfile  as 
being  affected  by  the  degradation  effects.  The  illustration  in  Figure  17  was  used  as  a 
guide  throughout  the  detection  distance  analysis.  The  figure  helped  to  track  which 
sensors  were  influenced  by  which  TAO  degradation  effects.  The  figure  illustrates  that 
Satl,  Sat2,  the  GlobalHawk,  and  the  Predator _UAV were  all  coded  in  the  Kosovo  warfile 
as  being  effected  by  both  weather  and  terrain  effects,  while  the  Elint_SAT  was  affected 
only  by  the  weather  TAO  and  the  JSTARS  was  affected  only  by  the  terrain  TAO.  As 
mentioned  in  Chapter  3,  the  weather  and  terrain  effects  influence  specific  performance 
attributes.  Weather  affects  platform  speed,  sensor  probability  of  detection,  weapon 
probability  of  kill,  and  communications  reliability.  Terrain  affects  platform  speed,  sensor 
range,  weapon  range,  and  communications  range.  The  degradation  effects  are 
implemented  to  degrade  network  performance  in  the  two  distinct  TAO  regions  according 
to  their  respective  influence  on  performance  attributes. 
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Figure  17.  Blue  Force  Sensors  Affected  by  Network  Degradation  Effects 


Graphical  trends  seen  in  average  detection  distance  plots  for  a  single  run  of  the 
Kosovo  scenario  helped  to  focus  the  subsequent  analysis  of  data  gained  from  thirty 
replications.  Figure  18  is  a  SEAS  Post  Processor  plot  of  average  detection  distance  data 
from  one  run  of  the  full  effects  case.  The  agents  listed  on  the  “Sensors”  and  “Targets” 
axes  are  not  all  inclusive  for  the  sake  of  space  and  clarity  of  reading  in  the  figure. 
Therefore,  the  hash  marks  on  the  “Sensors”  axis  listing  F15E#1,  F15E#3,  and  F15E#6, 
for  example,  represent  the  whole  group  of  F-15E  agents.  Similarly,  the  specific  listings 
on  the  “Targets”  axis  for  individual  members  of  the  RedSA6,  Serb_Armor,  and  Ktr actor 
units  are  not  representative  of  those  types  of  agents  for  that  region  of  the  axis.  Since 
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there  is  only  one  JSTARS  in  the  scenario,  its  hash  mark  on  the  “Sensors”  axis  correctly 
lines  up  with  the  JSTARS  row  of  average  detection  distances  versus  various  type  of 
target. 


Figure  18.  Average  Detection  Distance  Versus  Various  Targets  and  Sensors 


Several  trends  and  points  of  interest  can  be  gleaned  concerning  the  behavior  of 
agents  within  the  Kosovo  scenario  from  this  plot.  First  of  all,  the  JSTARS  is  the  most 
active  and  effective  Blue  force  sensor,  clearly  seeing  the  most  Red  targets  and  at  the 
farthest  average  ranges,  anywhere  from  20  to  120  kilometers.  Also,  the  F-15’s  are  fairly 
effective  at  detecting  Red  armor  and  surface-to-air  threats,  but  not  nearly  to  the  range  of 
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the  JSTARS.  Last,  the  Red  radar  vans  are  detecting  the  F-15’s  fairly  consistently  and 
from  distances  of  20  to  100  kilometers,  which  is  much  farther  away  than  the  F-15’s  are 
seeing  their  targets,  although  the  F-15’s  can  be  cued  by  other  Blue  ISR  assets. 

Seeing  these  detection  trends  from  single  run  output  data  was  very  helpful  in 
better  approaching  the  thirty  runs  analysis.  From  this  single  run  analysis,  it  was  learned 
which  sensor  platforms  would  be  most  worth  focusing  comparative  performance  analysis 
on  for  the  three  degraded  scenario  cases  versus  the  baseline  case.  Also,  knowing  which 
targets  were  being  detected  by  which  sensors  helped  to  provide  a  fuller  understanding  of 
what  types  of  detections  the  more  aggregated  data  for  thirty  runs  was  truly  representing. 

4.3.2  Thirty  Runs  Analysis  -  Four  Cases  Output  Comparison 

The  second  phase  of  analysis  conducted  for  detection  distances  of  the  Kosovo 
scenario  was  to  compare  average  detection  distance  outputs  from  thirty  runs  of  the 
baseline  case,  which  has  no  weather  or  terrain  effects,  versus  average  detection  distance 
outputs  from  thirty  runs  of  the  three  states  of  network  degradation  (represented  as  the 
application  of  weather  only,  terrain  only,  and  weather  and  terrain  effects  combined).  The 
goal  of  this  analysis  is  to  determine  whether  the  difference  between  case  outputs  is 
statistically  significant.  A  paired-/  confidence  interval  approach  is  selected  as  the 
statistical  tool  to  test  for  this  difference,  with  the  key  indicator  of  statistical  difference 
being  whether  or  not  zero  is  included  in  the  confidence  interval  for  difference  in  outputs. 
If  zero  is  included  in  the  confidence  interval,  then  there  cannot  be  a  conclusion  of 
statistical  difference  between  the  two  model  outputs  being  compared. 
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The  procedure  of  the  paired-/  confidence  interval  approach  involves  first  defining 


the  variable  Z  as 


zj=xj~yj ’ 


where  X  j  is  the  random  variable  average  output  from  the  baseline  model.  For  the 
analysis  of  Kosovo  scenario  outputs,  X .  represents  average  output  for  the  baseline  case, 
where  no  terrain  and  weather  effects  are  present.  7.  is  the  random  variable  output  from 

the  model  against  which  the  baseline  is  being  compared.  In  the  context  of  the  Kosovo 
scenario,  Yj  represents  the  three  degraded  cases  of  terrain  effects  only,  weather  only,  and 

combination  of  weather  and  terrain.  The  expected  value  of  the  Z  '  s  is 


Z(n)  = 


y"  z 

Lu  7=1  J 


The  approximate  100(1-  a )  percent  confidence  interval  is  defined  by 


Z('7)±W 


Var\Z(n)\ 


where  tn_u_a/  is  the  test  statistic  and  Var[Z{n)]  is  defined  as 


Var[Z(n )]  = 


0/-!) 
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Table  1.  Satellites  Paired-f  Test  Detection  Distance  Analysis 


Satellite  #1 

Difference  Between 

Baseline  and: 

Z(ri) 

95  %  Confidence 
Interval 

Statistical 

Difference? 

Percentage  Change 
from  Baseline: 

Full  Effects 

178.30 

(166.38,190.23) 

Yes 

-13.75 

Terrain  Only 

174.57 

(165.88,  183.26) 

Yes 

-13.46 

Weather  Only 

15.91 

(3.18,  28.63) 

Yes 

-1.23 

Satellite  #2 

Difference  Between 
Baseline  and: 

Z(n) 

95  %  Confidence 
Interval 

Statistical 

Difference? 

Percentage  Change 
from  Baseline: 

Full  Effects 

176.66 

(164.02,  189.30) 

Yes 

-13.80 

Terrain  Only 

164.00 

(154.65,  173.35) 

Yes 

-12.81 

Weather  Only 

18.05 

(0.36,  35.75) 

Yes 

-1.41 

Table  1  shows  Z(n )  and  the  95%  confidence  interval  ( a  =  0.05)  for  Satl.  The 
full  paired-/  test  results  and  analysis  are  listed  in  Appendix  A.  The  table  lists  whether  or 
not  statistically  significant  differences  exist  between  each  degraded  case  and  the  baseline 
model  case  for  average  detection  distance  outputs  and  also  the  percentage  change  in  the 
average  detection  distance  from  the  baseline  case.  Table  1  illustrates  that  both  satellites’ 
average  detection  distance  ranges  are  clearly  reduced,  especially  in  the  full  effects  and 
terrain  only  cases.  It  is  a  bit  surprising  that  the  weather  case  did  not  hinder  the  average 
detection  distance  more  severely  for  both  satellites.  This  could  be  due  to  the  fact  that 
both  satellites  are  detecting  targets  less  frequently  in  the  weather  case,  as  the  weather 
factor  of  zero  in  the  TPL  eliminates  line  of  sight  target  viewing  for  each  satellite. 
Detections  are  still  possible  for  the  satellites  on  the  edges  of  the  weather  TAO,  but  a 
smaller  number  of  detections  may  be  limiting  observance  of  the  true  degradation  affect  in 
the  weather  only  case. 
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Table  2.  F-15  Squadron  Paired-tTest  Detection  Distance  Analysis 


F-15E#1 

Difference  Between 
Baseline  and: 

Z(n) 

95  %  Confidence 
Interval 

Statistical 

Difference? 

Percentage  Change 
from  Baseline: 

Full  Effects 

-0.44 

(-6.19,  5.31) 

No 

1.05  i 

Terrain  Only 

-6.56 

(-11.71,-1.41) 

Yes 

15.78  1 

Weather  Only 

1.56 

(-4.75,  7.87) 

No 

-3.75 

F-15E#4 

Difference  Between 

Baseline  and: 

Z{n) 

95  %  Confidence 
Interval 

Statistical 

Difference? 

Percentage  Change 
from  Baseline: 

Full  Effects 

0.38 

(-8.22,  8.97) 

No 

-0.92 

Terrain  Only 

-2.15 

(-8.45,  4.16) 

No 

5.20 

Weather  Only 

3.07 

(-2.83,  8.97) 

No 

-7.42 

All  6  F-15's  Together 

Difference  Between 

Baseline  and: 

Z(») 

95  %  Confidence 
Interval 

Statistical 

Difference? 

Percentage  Change 
from  Baseline: 

Full  Effects 

0.49 

(-3.39,  4.37) 

No 

-1.18 

-3.07 

(-7.05,  0.91) 

No 

7.38  j 

i  Weather  Only 

0.62 

(-2.99,  4.23) 

No 

-1.49  ; 

Table  2  shows  Z(n )  ,  the  95%  confidence  intervals,  and  percentage  changes  in 
average  detection  distances  versus  the  baseline  case  for  F-15EU1,  F-15E#4,  and  the  F-15 
squadron  as  a  whole.  This  table  illustrates  that,  except  for  the  F-15#l  comparison  of  the 
baseline  with  the  terrain  only  effect,  there  is  no  statistical  difference  between  the  average 
F-15  squadron  detection  distances  for  all  of  the  three  case  comparison  variations  versus 
the  base  case.  This  is  essentially  the  expected  result  since  the  F-15’s  are  not  coded  in  the 
Kosovo  wariile  as  being  affected  by  the  weather  or  terrain  TAO.  However,  the 
improvement  in  F-15#F s  average  detection  distance  in  the  case  where  only  terrain 
effects  are  applied  is  not  clearly  understood.  Perhaps  this  improvement  in  average 
detection  distance  is  due  to  the  fact  that  the  satellites’  detection  distances  are  severely 
hampered  and  therefore  F-15#l  is  not  able  to  rely  on  cueing  information  from  the 
satellites,  but  rather  must  more  actively  seek  out  targets  on  its  own.  F-15#l  is  the  first  F- 
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15  to  deploy  from  the  Blue  base  and  it  is  able  to  relay  this  infonnation  on  to  the  rest  of 
the  squadron,  which  rely  on  both  the  satellites’  and  F-15#F s  detection  information  to 
guide  them  to  targets.  This  may  be  why  F-15#V s  average  detection  distance  undergoes 
this  change  for  the  terrain  only  case  while  F-15#F s  average  detection  distance,  as  well  as 
that  of  the  squadron  as  a  whole,  are  not  significantly  different.  In  summary  of  the  data 
analysis  presented  in  Tables  1  and  2,  terrain  and  weather  effects  are  seen  to  significantly 
affect  the  NCW  physical  domain  metric  of  detection  distance  for  the  satellites  in  the 
Kosovo  scenario,  but  not  for  the  F-15’s.  Due  to  their  respective  coding  in  the  warfile  as 
to  how  the  terrain  and  weather  affects  each  platfonn,  this  is  the  expected  outcome. 


Table  3.  JSTARS  and  Global  Hawk  Paired-f  Test  Detection  Distance  Analysis 


1  JSTARS  | 

Difference  Between 
Baseline  and: 

Z(n ) 

95  %  Confidence 
Interval 

Statistical 

Difference? 

Percentage  Change 
from  Baseline: 

Full  Effects 

0.21 

(-2.09,  2.51) 

No 

-0.33 

Terrain  Only 

0.28 

(-2.48,  3.05) 

No 

-0.44 

Weather  Only 

-0.63 

(-3.05,  1.79) 

No 

0.98 

Global  Hawk 

Difference  Between 
Baseline  and: 

Z(n) 

95  %  Confidence 
Interval 

Statistical 

Difference? 

Percentage  Change 
from  Baseline: 

Full  Effects 

-0.06 

(-0.18,  0.05) 

No 

0.24 

Terrain  Only 

-0.06 

(-0.12,  0.00) 

No 

0.24 

Weather  Only 

0.13 

(0.03,  0.23) 

Yes 

-0.53  1 

Similar  paired-/  confidence  interval  analysis  was  also  perfonned  for  the  JSTARS 
and  GlobalFIawk  agents.  Table  3  summarizes  the  results  of  this  analysis.  The  results  in 
Table  3  show  that  in  five  case  comparisons  out  of  six  for  the  JSTARS  and  GlobalFIawk, 
there  were  no  statistically  significant  differences  in  the  average  detection  distances  for 
each  platfonn.  In  the  comparison  for  the  GlobalFIawk  in  the  case  of  baseline  versus 
weather  only  effects,  even  though  this  change  was  statistically  significant  according  to 
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the  paired-t  confidence  interval,  the  percentage  decrease  in  average  detection  distance  of 
0.53  percent  is  arguably  not  practically  significant.  The  lack  of  statistically  significant 
differences  across  the  case  comparisons  for  the  average  detection  distances  of  the 
JSTARS  and  GlobalHawk  is  an  unexpected  result.  Both  platforms  exhibit  higher  degrees 
of  sensing  activity  in  all  scenarios  over  all  four  cases  than  any  other  platfonn. 

The  JSTARS  is  coded  as  being  affected  by  the  terrain  TAO  only,  so  the  lack  of 
difference  in  performance  for  the  weather  effects  only  case  is  understandable.  However, 
it  would  stand  to  reason  that  an  observable  difference  in  sensor  detection  distance  range 
would  be  seen  for  the  full  effects  and  terrain  only  cases.  It  is  not  clear  why  the  expected 
differences  in  output  are  not  observed.  The  same  holds  true  for  the  GlobalHawk, 
especially  in  light  of  the  fact  that  this  platform  is  coded  as  being  affected  by  both  the 
terrain  and  the  weather  TAO.  And  yet,  there  is  no  statistical  decrease  of  sensor  distance 
range  for  any  of  the  three  case  comparisons  for  this  platform,  except  for  the  weather  only 
case  whose  increase  in  range,  while  very  unexpected,  is  not  of  a  magnitude  to  be 
considered  practically  significant. 

One  possible  conclusion  that  can  be  drawn  from  this  lack  of  statistically 
significant  difference  for  the  JSTARS  and  GlobalHawk  is  that  average  detection  distance 
may  not  be  a  reliable  metric  within  SEAS  by  which  to  measure  the  physical  domain  for 
sensing  platforms  other  than  satellites.  Apparently,  the  degradation  effects  of  the  weather 
and  terrain  TAO  are  having  significant  affects  for  the  long-range  sensing  activities  of  the 
satellites  but  not  for  the  relatively  closer  range  detections  of  the  JSTARS  and 
GlobalHawk. 


58 


4.4  Information  Domain  Analysis 


4.4.1  Preliminary  Multiple  Run  Analysis 

The  metric  selected  for  the  information  domain  in  the  Kosovo  scenario  was  a 
performance  measurement  of  the  networks’  communication  channels.  Specifically,  the 
number  of  messages  handled  by  each  channel  was  analyzed  for  key  platforms  of  the  Blue 
Force.  The  focus  was  on  determining  the  affect  of  regional  TAO  degradation  on  each 
channels’  ability  to  handle  and  transfer  messages  pertaining  to  target  detections,  agent 
orders,  and  a  few  variable  types  of  messages.  All  three  types  of  messages  are  tracked  in 
SEAS  for  each  channel  specified  in  the  TPL  and  designated  in  the  communications 
output  file  as  the  channel  name  followed  by  _Sit_,  for  situation  report  (i.e.  target 
sighting),  _Var_,  for  broadcast  variables  (which  can  be  various  message  types  such  as 
target  priority  arrays),  and  _Ord_,  for  orders  and  command  messages. 
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Figure  19.  AOC  Communication  with  Group  &  Air  Assets  (DeStefano,  2004:3-10) 


Figure  19  is  a  graphic  illustration  of  the  Blue  Force  communications  network 
from  DeStefano’s  work.  The  figure  shows  that  the  TAC_Air_ORD(6,3)  and 
TAC_Air_Ord(l ,  1)  communications  lines  provide  a  critical  link  between  the  AOC  and 
several  key  Blue  platforms,  including  the  F-15  squadron,  F-16  squadron,  and 
Blu  Cruiser,  which  is  a  Navy  carrier  agent  that  launches  the  TOMAHAWK  Land  Attack 
Missile  (TLAM).  Analysis  of  message  loading  and  activity  across  all  channels 
conducted  for  this  thesis  effort  confirms  that  the  TAC  Air  communication  device’s 
primarily  used  channel,  TacAirQ  Sit,  is  one  of  the  most  highly  active  channels  in  the 
scenario.  It  relays  target  sightings  to  the  aforementioned  platforms. 

As  a  first  look  in  trying  to  appropriately  measure  the  communications  output  data 
for  individual  channel  loading,  one  run  of  the  scenario  was  analyzed  to  look  at  both  the 
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one-run  total  number  of  messages  and  the  average  number  of  messages  handled  by  each 
channel  per  one  minute  time-step.  Table  4  shows  the  results  of  this  preliminary  analysis. 

Only  the  data  for  number  of  messages  removed  from  each  respective  channel  is  listed  in 
the  table  since  this  value  is  most  indicative  of  the  activity  on  the  channel  and  the  message 
loading  for  each  channel.  It  is  identical  to  the  output  data  for  the  number  of  messages 
added  to  each  channel.  The  number  of  messages  currently  on  a  channel  is  tracked  by 
SEAS,  but  this  metric  is  not  used  because  the  amount  of  time  that  each  channel 
broadcasts  a  batch  of  messages  varies  according  to  its  delay  time  attribute.  Therefore, 
channels  having  a  longer  programmed  delay  time  would  show  higher  average  total 
message  counts  and  average  messages  per  time-step  counts  than  agents  with  shorter 
broadcast  times  and  unequal  channel  performance  comparisons  would  be  made.  Using 
the  number  of  messages  removed  from  each  channel  levels  the  playing  field  and  works  to 
normalize  the  data  for  each  channel. 
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Table  4.  All  Active  Communication  Channels  Analysis 


COMPARISON  OF  ALL  ACTIVE  CHANNELS’  TOTAL  MESSAGE  COUNT  AND 
AVERAGE  MESSAGE  COUNT  PER  MINUTE  TIME-STEP  FOR  ONE  RUN 


BASELINE  -  No  Effects 

Full  Effects 

BASELINE  minus  Full  Effects 

Channel 

Count 

Average 

Count 

Average 

Count  Diff 

Average  Diff 

JSTARSQ  Sit  Rem 

2262.00 

0.51 

3728.00 

0.61 

(1466.00) 

(0.10) 

GShipQ  Sit  Rem 

1985.00 

0.43 

424.00 

0.10 

1561.00 

0.33 

GHQ  Sit  Rem 

3362.00 

0.68 

833.00 

0.15 

2529.00 

0.53 

SBRQ  Sit  Rem 

22302.00 

4.90 

18781.00 

3.23 

3521.00 

1.67 

TacAirQ  Sit  Rem 

22302.00 

4.90 

18781.00 

3.23 

3521.00 

1.67 

GShip  OrdQ  Ord  Rem 

1.00 

0.00 

1.00 

0.00 

0.00 

0.00 

Air  OrdQ  Ord  Rem 

3.00 

0.00 

3.00 

0.00 

0.00 

0.00 

ImSatQ  Sit  Rem 

64.00 

0.01 

14.00 

0.00 

50.00 

0.01 

ElintSATQ  Sit  Rem 

233.00 

0.06 

216.00 

0.06 

17.00 

0.00 

SOF  OrdQ  Ord  Rem 

342.00 

0.21 

207.00 

0.13 

135.00 

0.09 

SOF  Sat  PhQ  Sit  Rem 

336.00 

0.05 

311.00 

0.06 

25.00 

(0.01) 

RTac  OrdQ  Var  Rem 

11342.00 

2.14 

11342.00 

2.12 

0.00 

0.03 

RTac  OrdQ  Ord  Rem 

11198.00 

1.93 

11578.00 

1.99 

(380.00) 

(0.06) 

RIADSQ  Sit  Rem 

827.00 

0.18 

1205.00 

0.25 

(378.00) 

(0.06) 

RSRTQ3  Sit  Rem 

371.00 

0.20 

353.00 

0.20 

18.00 

0.00 

KSHQ  Sit  Rem 

218.00 

0.04 

237.00 

0.04 

(19.00) 

0.00 

KSHQ  Ord  Rem 

283.00 

0.05 

249.00 

0.05 

34.00 

0.00 

KBellQ  Sit  Rem 

3652.00 

0.45 

6742.00 

0.78 

(3090.00) 

(0.33) 

KBellQ  Ord  Rem 

24.00 

0.00 

24.00 

0.00 

0.00 

0.00 

Terrain  Only 

BASELINE  minus  Terrain  Only 

Count 

Average 

Count  Diff 

Average  Diff 

JSTARSQ  Sit  Rem 

2260.00 

0.46 

2.00 

0.05 

GShipQ  Sit  Rem 

1800.00 

0.34 

185.00 

0.09 

GHQ  Sit  Rem 

3197.00 

0.56 

165.00 

0.12 

SBRQ  Sit  Rem 

21321.00 

4.11 

981.00 

0.79 

TacAirQ  Sit  Rem 

21321.00 

4.11 

981.00 

0.79 

GShip  OrdQ  Ord  Rem 

1.00 

0.00 

0.00 

0.00 

Air  OrdQ  Ord  Rem 

3.00 

0.00 

0.00 

0.00 

ImSatQ  Sit  Rem 

21.00 

0.01 

43.00 

0.01 

ElintSATQ  Sit  Rem 

205.00 

0.04 

28.00 

0.02 

SOF  OrdQ  Ord  Rem 

567.00 

0.13 

(225.00) 

0.08 

SOF  Sat  PhQ  Sit  Rem 

131.00 

0.04 

205.00 

0.01 

RTac  OrdQ  Var  Rem 

13448.00 

2.26 

(2106.00) 

(0.12) 

RTac  OrdQ  Ord  Rem 

7138.00 

1.73 

4060.00 

0.20 

RIADSQ  Sit  Rem 

1250.00 

0.20 

(423.00) 

(0.01) 

RSRTQ3  Sit  Rem 

367.00 

0.09 

4.00 

0.10 

KSHQ  Sit  Rem 

303.00 

0.04 

(85.00) 

0.00 

KSHQ  Ord  Rem 

310.00 

0.05 

(27.00) 

0.00 

KBellQ  Sit  Rem 

3635.00 

0.42 

17.00 

0.03 

KBellQ  Ord  Rem 

24.00 

0.00 

0.00 

0.00 

Weather  Only 

BASELINE  minus  Weather  Only 

Count 

Average 

Count  Diff 

Average  Diff 

JSTARSQ  Sit  Rem 

1077.00 

0.48 

1185.00 

0.04 

GShipQ  Sit  Rem 

131.00 

0.09 

1854.00 

0.33 

GHQ  Sit  Rem 

180.00 

0.16 

3182.00 

0.53 

SBRQ  Sit  Rem 

5328.00 

2.65 

16974.00 

2.26 

TacAirQ  Sit  Rem 

5328.00 

2.65 

16974.00 

2.26 

GShip  OrdQ  Ord  Rem 

1.00 

0.00 

0.00 

0.00 

Air  OrdQ  Ord  Rem 

3.00 

0.00 

0.00 

0.00 

ImSatQ  Sit  Rem 

27.00 

0.01 

37.00 

0.01 

ElintSATQ  Sit  Rem 

172.00 

0.04 

61.00 

0.01 

SOF  OrdQ  Ord  Rem 

189.00 

0.08 

153.00 

0.13 

SOF  Sat  PhQ  Sit  Rem 

512.00 

0.05 

(176.00) 

0.00 

RTac  OrdQ  Var  Rem 

13367.00 

2.19 

(2025.00) 

(0.05) 

RTac  OrdQ  Ord  Rem 

7118.00 

1.86 

4080.00 

0.07 

RIADSQ  Sit  Rem 

765.00 

0.18 

62.00 

0.00 

RSRTQ3  Sit  Rem 

369.00 

0.11 

2.00 

0.09 

KSHQ  Sit  Rem 

264.00 

0.04 

(46.00) 

0.00 

KSHQ  Ord  Rem 

256.00 

0.05 

27.00 

0.00 

KBellQ  Sit  Rem 

9828.00 

0.62 

(6176.00) 

(0.17) 

KBellQ Ord Rem 

24.00 

0.00 

0.00 

0.00 
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Table  4  gives  a  good  indicator  of  not  only  which  channels  are  handling  the 
highest  message  loads,  but  also  provides  a  good  illustration  of  which  channels  are  most 
affected  by  the  degradation  effects.  Values  which  are  bold  and  listed  in  parenthesis 
represent  negative  values  and  therefore,  these  are  the  cases  and  channels  which  yielded 
higher  levels  of  either  total  message  count  and/or  average  message  count  per  one  minute 
time-step  and  therefore  actually  saw  higher  activity  for  that  respective  degraded  case  and 
channel  combination  relative  to  the  baseline  case  perfonnance. 

Channels  of  interest  which  saw  this  increase  in  activity  for  both  total  message 
count  and  average  message  count  for  at  least  one  case  versus  the  baseline  are  shaded 
since  these  channels  exhibit  unexpected  behavior  for  the  degraded  cases.  These  six 
channels  of  interest  are  JSTARSQSit,  RTacOrdQOrd,  RIADSQSit,  RSRTQ3_Sit, 
KSHQOrd,  and  KBellQSit.  JSTARSQ  Sit  is  a  channel  that  relays  target  sighting 
information  from  the  JSTARS  agent.  RTac  OrdQ  Ord  carries  orders  for  the  Red  Force. 
RIADSQ  Sit  is  a  channel  used  by  the  RedlADSnet  device,  which  is  held  by  the 
RedSA61Tel,  RedSA62Tel,  RedSA61RadarVan,  and  RedSA62RadarVan  vehicles. 

KSHQ  Ord  and  KBellQ  Sit  are  used  by  the  Brown  Kosovar  agents  as  distress  emitting 
“channels”  on  which  to  shout  commands  to  each  other  and  ring  bells  to  signal  attack  by 
the  Red  Serbian  Force.  Of  these  six  channels  of  interest  based  on  their  communications 
data  improvements  in  the  degraded  cases,  only  JSTARSQ  Sit  is  of  particular  interest 
since  this  is  a  highly  active  channel  of  the  Blue  Force  and  since  it  is  used  by  a  vehicle, 
the  JSTARS,  that  is  affected  by  the  terrain  TAO,  whereas  the  Red  and  Brown  forces  are 
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not  affected  by  either  the  terrain  or  the  weather  TAO.  Thus,  of  these  six  channels,  only 
JSTARSQ  Sit  warrants  further  analysis. 

4.4.2  Average  Message  Loading  of  Active  Channels 

The  problem  with  the  foregoing  analysis  method  for  the  communications  data  is 
that  the  zero  values  for  time-steps  when  no  messages  are  removed  from  a  channel  tend  to 
distort  the  calculated  averages.  The  legitimacy  of  this  analysis  technique  is  also 
weakened  by  the  fact  that  the  output  data  stream  from  a  single  run  is  not  independent.  A 
closer  look  and  more  intensive  analysis  approach  is  required  in  order  to  detennine  the 
values  for  a  more  appropriate  measure,  which  would  be  the  average  message  load 
handled  by  each  channel  only  during  the  times  when  that  channel  is  holding  a  batch  of 
messages.  Once  again,  the  number  of  messages  removed  field  from  the  standard  SEAS 
communication  output  file  will  be  utilized.  The  preliminary  analysis  conducted  over  all 
time-steps  for  one  replication  was  used  as  a  guide  for  determining  the  communication 
channels  to  focus  on  for  analysis  of  average  channel  load  only  when  that  channel  is 
active.  Only  the  baseline  case  and  full  effects  case  were  analyzed  for  the  sake  of 
comparison  due  to  the  considerable  amount  of  time  required  to  extract  the  desired 
information  from  the  SEAS  raw  data  communication  output  files.  Table  5  presents 
average  active  channel  usage  data  for  the  top  five  most  active  channels  in  the  Kosovo 
model. 
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Table  5.  Average  Active  Channel  Usage  for  One  Simulation  Run 


i  BASELINE  -  NO  EFFECTS  CASE  | 

Channel 

JSTARSQ_Sit 

SBRQ_Sit 

TacAirQ_Sit 

RTac_  OrdQ_  Var 

RTac_OrdQ_Ord 

Number  of  Active  Minute 
Time-Steps 

196.00 

324.00 

179.00 

513.00 

1132.00 

Average  Message  per 
Active  Time-Step 

7.18 

64.82 

117.34 

27.16 

14.77 

FULL  EFFECTS  CASE 

Channel 

JSTARSQ_Sit 

SBRQ_Sit 

TacAirQ_Sit 

RTac_OrdQ_  Var 

RTac_OrdQ_Ord 

Number  of  Active  Minute 
Time-Steps 

352.00 

293.00 

187.00 

525.00 

882.00 

Average  Message  per 
Active  Time-Step 

10.43 

58.02 

90.91 

23.60 

14.88 

This  table  shows  mixed  results  as  far  as  the  usefulness  of  the  average  active  time- 
step  measure.  The  JSTARSQSit  channel  has  already  been  discussed.  The  SBRQSit  is 
one  of  the  primary  channels  used  by  the  CAOC  to  relay  target  sighting  infonnation.  The 
TacAirQ_Sit  has  also  been  previously  discussed.  RTac  OrdQ  Var  and  RTac  OrdQ  Ord 
are  command  channels  used  by  the  Red  Force  to  relay  various  types  of  orders 
information  to  the  Red  units  and  vehicles. 

There  seems  to  be  no  clear  or  consistent  pattern  of  either  decrease  or  increase  for 
the  average  number  of  active  time-steps  across  these  channels  and  cases.  The  average 
number  of  messages  per  active  time-step  measure  seems  to  be  more  indicative  of  a  real 
trend,  in  that  three  out  of  five  channels  show  a  drop  in  average  active  usage  from  the 
baseline  case  to  the  full  effects  case.  A  decrease  in  message  load  for  the  full  effects  case 
is  the  expected  result,  especially  for  the  Blue  Force  channels  of  JSTARSQ  Sit,  SBRQ  Sit, 
and  TacAirQ_Sit.  JSTARSQ  Sit  activity  should  decrease  due  to  a  reduced  number  of 
target  sightings  because  of  the  terrain  effect  that  the  JSTARS  agent  is  coded  as  being 
affected  by.  However,  this  is  not  the  case,  as  the  activity  on  JSTARSQ  Sit  actually 
increases  for  the  full  effects  case.  The  message  load  per  time-step  of  SBRQ  Sit  and 
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TacAirQ_Sit  decreases  for  the  full  effects  case,  as  would  be  expected  since  the  CAOC 
should  have  less  target  sighting  messages  to  relay,  especially  from  the  satellites  since 
their  target  detection  frequency  and  range  was  significantly  affected  by  the  weather  and 
terrain  effects.  RTac  OrdQ  Var  sees  a  decrease,  from  27.16  to  23.60,  in  average 
message  load  per  active  time-step,  as  might  be  expected  since  the  Red  Force  behaves 
largely  in  a  reactive  way  to  Blue  Force’s  activities.  The  presence  of  full  degradation 
effects  tends  to  reduce  the  overall  activity  of  Blue  and  consequently  tends  to  reduce  the 
reactionary  activity  of  Red.  RTac  OrdQ  Ord  carries  a  slightly  higher  number  of  average 
messages  per  active  time-step  for  the  full  effects  case,  but  the  increase  is  less  than  one 
percent  (from  14.77  to  14.88,  a  0.74%  increase)  and  therefore  arguably  not  practically 
significant. 

4.4.3  Average  Message  Loading  Over  Time 

A  final  approach  taken  to  determine  an  appropriate  and  usable  infonnation 
domain  metric  which  can  be  gleaned  from  the  SEAS  communication  output  data  involves 
plotting  the  overall  average  message  load  for  the  top  four  active  channels  over  ten  ten- 
hour  segments  of  one  simulation  run.  The  resulting  plots  are  illustrated  in  Figure  20  and 
2 1 .  The  average  number  of  messages  per  ten-hour  time  block  is  calculated  over  all  60 
minute  time-steps  for  the  baseline  and  full  effects  cases  using  the  same  starting  random 
number  seed.  There  was  no  adjustment  made  to  filter  out  time-steps  when  the  channels 
are  broadcasting  zero  messages.  Four  out  of  five  of  the  communications  channels 
selected  for  the  previous  phase  of  analysis  are  presented  in  these  plots.  SBRQ  Sit  was 
excluded  on  these  plots  because  this  channel’s  average  message  activity  per  ten-hour 
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time  segment  is  exactly  the  same  as  the  TacAirQ_Sit  channel’s  average  number  of 
messages  and  this  holds  true  for  both  the  baseline  case  and  full  effects  case. 


Baseline  (No  Effects)  Case  -  Average  Message  Load  per 
10  Hours  of  Kosovo  Scenario  for  Top  Four  Active  Channels 


0-10  11-20  21-30  31-40  41-50  51-60  61-70  71-80  81-90  91-100 


— JSTARSQSit 
-A—  TacAirQSit 
X  RTacOrdQVar 
X  RTac  OrdQ  Ord 


Hour  Segment 


Figure  20.  Baseline  Case  Average  Message  Load  per  10-hour  Segment 


Full  Effects  Case  -  Average  Message  Load  per  10  Hours 
of  Kosovo  Scenario  for  Top  Four  Active  Channels 


-  JSTARSQ_Sit 

-  TacAirQ_Sit 

-  RTac_OrdQ_Var 
-RTac  OrdQ  Ord 


Figure  21.  Full  Effects  Case  Average  Message  Load  per  10-hour  Segment 
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A  few  trends  can  be  seen  in  the  average  communication  loading  for  these  top  four 
active  channels.  A  pattern  of  relatively  high  message  activity  on  TacAirQ  Sit  for 
approximately  the  first  20  hours,  then  decrease  up  until  approximately  50  hours,  followed 
by  a  rise  until  about  the  70  hour  mark  and  fall  after  that,  holds  true  for  both  cases.  These 
two  distinct  phases  of  communication  activity  match  up  closely  with  DeStefano’s 
findings  concerning  phases  of  war  for  the  Kosovo  scenario.  Figure  22  illustrates  these 
two  phases  in  terms  of  number  of  kills. 


Baseline:  All  Agent  Kills  Over  Time 


Time(h) 


RED.SUV 

R  ED_Serb_G  roundtgt  1  _Ve  h 
RED_RedSA62RadarVan 
RED_RodSA6 1  RadarV  an 
RED_RedSA3RadarVan 
GRY_Khouse 
BLU_F15E 


Figure  22.  Two  Phases  of  the  Kosovo  War  Based  on  Kill  Data  (DeStefano,  2004:4-3) 


Phase  I,  origin  of  the  war  to  approximately  48  hours,  is  considered  a  SEAD  phase. 
Phase  II,  from  approximately  48  hours  on,  is  considered  as  intervention  of  killing  on  the 
ground.  This  phase  is  highlighted  by  a  large  distribution  of  Kosovar  kills  as  opposed  to 
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other  activities  occurring  during  this  time  (DeStefano,  2004:4-2).  The  plots  for  both 
cases  portraying  the  average  number  of  messages  per  ten-hour  segment  match  up  nicely 
with  these  two  phases  of  war  and  the  generally  lower  number  of  messages  on 
TacAirQ_Sit  for  the  full  effects  case  matches  the  expected  outcome  for  this  degraded 
case. 

The  trends  are  not  quite  as  easy  to  observe  for  data  on  the  Red  channels, 
RTacOrdQVar  and  RTacOrdQOrd.  RTacOrdQVar  follows  a  similar  nearly  flat- 
line  pattern  in  both  cases  and  generally  holds  a  slightly  lower  number  of  average 
messages  per  ten-hour  segment  (ranges  from  0.18  to  0.83  for  ten-hour  segments  where 
the  no  effects  average  is  greater  than  the  full  effects  average).  RTac  OrdQ  Ord  follows 
essentially  the  same  exact  progression  of  values  for  each  case  over  the  first 
approximately  50  hours,  but  after  that  the  average  number  of  messages  on  this  channel 
for  the  full  effects  case  shoots  up  to  nearly  twice  the  amount  as  seen  in  the  no  effects 
baseline.  The  general  trend  of  RTacOrdQOrd"  s  message  activity  for  both  cases  seems 
to  match  the  two-phase  pattern  seen  by  DeStefano ’s  kill  data  analysis  and  by  the  message 
activity  line  plotted  in  each  case  for  TacAirQ_Sit. 

Activity  for  JSTARSQSit  for  both  cases  also  roughly  matches  the  two-phases  of 
war  trend.  However,  the  average  message  load  on  JSTARSQ  Sit  again  appears  to 
increase  for  the  full  effects  case,  which  again  is  an  unexpected  given  that  the  JSTARS  is 
coded  as  being  affected  by  the  terrain  TAO.  Terrain  effects  should  result  in  a  drop  of  the 
communications  reliability  and  hence  a  lower  number  of  average  messages  being  relayed, 
but  this  is  not  what  the  output  data  is  showing.  Just  as  was  the  case  in  the  detection 
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distance  analysis  for  the  JSTARS,  it  is  unclear  as  to  why  this  platform  is  perfonning  better 
in  the  degraded  cases. 

4.5  Cognitive  Domain  Analysis 

The  chosen  measure  for  the  cognitive  domain  of  NCW  is  the  somewhat  indirect 
metric  of  number  of  kills  (and,  consequently,  number  of  victim  deaths)  per  platform.  Kill 
data  is  representative  of  decision-making  behavior  because  the  recording  of  a  kill  in  the 
scenario  is  conclusive  evidence  of  the  outcome  resulting  from  a  decision  made  to  attack. 
The  kill  numbers  measure  the  “act”  part  of  the  OODA  (Observe,  Orient,  Decide,  Act) 
loop.  Unlike  the  physical  and  infonnation  domain  metrics,  the  outputs  for  kill  numbers 
used  to  measure  the  cognitive  domain  of  NCW  are  relatively  clear  and  definitive.  This 
section  illustrates  that,  in  general,  the  no  degradation  effects  (baseline)  case  is  the  best 
case  scenario  for  the  Blue  USAFE  Force  both  in  terms  of  higher  number  of  Red  killed 
and  lower  number  of  Brown  killed. 
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Figure  23.  Kill  Counts  for  Blue’s  Most  Active  Weapons  for  Ail  Four  Cases 


Figure  23  shows  the  cumulative  number  of  kills  over  thirty  runs  for  Blue’s  most 
active  weapons.  The  numbers  for  the  Gunship#4.Aircanonl 05  weapon  exhibit  a 
considerable  effect  of  the  degradation  states  on  the  final  outcome  of  this  agent’s  mission 
success  in  terms  of  enemy  agents  killed.  The  Blu_Cruiser#5.TLAM  weapon  also  exhibits 
the  trend  of  higher  kills  for  the  baseline  case  versus  the  three  degraded  cases.  The 
cumulative  kill  numbers  for  the  F-15s’  JSOW weapons,  however,  in  general  do  not  show 
the  same  clear  trends. 
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Cumulative  Victim  Count  Over  30  Runs 


Figure  24.  Blue  F-15  and  Red  Tel  Victim  Counts  for  All  Four  Cases 


Figure  24  shows  the  cumulative  victim  count  over  thirty  runs  of  each  case  for  the 
Blue  F-15’s  and  Red  SA6  Tels.  Similar  positive  trends  for  the  Blue  Force  are  seen  in  this 
victims  plot  as  were  seen  in  Figure  23.  Generally  less  Blue  agents  are  killed  in  the  no 
effects  baseline  case  as  compared  to  the  full  effects  case  (except  for  in  the  case  of 
F 1 5 _SEADSqdn#5 .F 1 5E#6).  The  results  for  F-15  losses  are  mixed  when  comparing  the 
baseline  case  with  the  terrain  only  and  weather  only  cases.  Red,  meanwhile,  has  higher 
losses  across  all  Red  SA6  Tels  in  the  no  effects  case  versus  the  baseline  case.  Similar  to 
the  F-15  losses,  the  results  for  Red  SA6  Tel  losses  are  mixed  when  comparing  the 
baseline  case  with  the  terrain  only  and  weather  only  cases. 
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Kosovars  Killed  Per  Case 


Figure  25.  Number  of  Kosovars  Killed  in  All  Four  Cases 


The  number  of  Kosovars  killed  over  the  various  effects  cases  is  shown  in  Figure 
25.  This  plot  also  shows  positive  trends  for  the  Blue  USAFE  Force,  whose  mission  is  to 
protect  the  Brown  Kosovar  Force  agents  and  minimize  the  number  of  Kosovars  killed.  It 
can  be  clearly  seen  from  the  various  kill  labels  that,  in  general,  less  Kosovar  agents  are 
dying  in  the  no  effects  baseline  case  as  opposed  to  the  three  degrading  effects  cases. 

Cumulative  kill  counts  over  30  runs  offer  rough  insight  into  behavior  and 
performance  within  the  Kosovo  scenario  over  the  four  degraded  cases.  However,  to  gain 
deeper  insight  into  the  true  behavior,  a  paired-/  confidence  interval  approach  will  once 
again  be  utilized.  The  procedure  used  here  is  similar  to  the  one  used  for  average 
detection  distance  analysis,  but  this  time  Xj  represents  total  kills  per  platform  for  the 
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baseline  case  and  Y.  represents  total  kills  per  platform  for  the  degraded  cases.  Results  of 

the  paired-t  confidence  interval  analysis  are  presented  in  Table  6.  The  full  paired-t  test 
results  and  analysis  are  listed  in  Appendix  A.  On  average  over  thirty  runs,  Blue  kills 
more  Red  SA  Tels  and  Radar  Vans  in  the  degraded  cases,  but  a  statistically  significant 
difference  is  not  found  at  a  95%  confidence  level.  Each  degraded  case  resulted  in  higher 
losses  for  the  number  of  Kosovar  houses  destroyed  by  Red.  While  no  statistically 
significant  difference  is  seen  for  the  terrain  and  weather  only  comparisons,  the  difference 
was  statistically  significant  at  a  95%  confidence  level  for  the  full  effects  versus  baseline 
comparison.  This  result  leads  to  the  conclusion  that  Blue  is  more  successful  at  achieving 
its  mission  of  saving  Kosovars  when  its  network  capability  of  sensing  and 
communicating  is  not  fully  degraded. 


Table  6.  Paired-tTest  Results  for  Red  and  Brown  Victim  Counts  Over  Thirty  Runs 


|  Blue  Kills  of  Red  SA  Tels  and  Radar  Vans  | 

Difference  Between 
Baseline  and: 

Z(n) 

95  %  Confidence 
Interval 

Statistical 

Difference? 

Full  Effects 

1.37 

(-0.32,  3.05) 

No 

Terrain  Only 

0.13 

(-1.22,  1.48) 

No 

Weather  Only 

0.83 

(-0.88,  2.54) 

No 

Kosovar  Houses  Destroyed  by  Red 

Difference  Between 
Baseline  and: 

Z(n) 

95  %  Confidence 
Interval 

Statistical 

Difference? 

Full  Effects 

-2.20 

(-4.33,  -0.07) 

Yes 

Terrain  Only 

-0.40 

(-1.50,  0.70) 

No 

Weather  Only 

-0.43 

(-1.75,  0.88) 

No 

4.6  Summary 

The  analysis  presented  in  this  chapter  demonstrates  that  the  task  of  developing 
appropriate  measures  for  NCW  within  the  context  of  a  SEAS  model  can  be  quite 
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challenging.  This  analysis  also  illustrates  that  detennining  whether  a  particular  metric  is 
fundamental  enough  to  serve  as  a  useful  measure  for  the  degree,  or  perfonnance,  of  NCW 
is  not  a  very  clear-cut  proposition.  Average  sensor  detection  distance  seemed  to  be  a 
fitting  and  effective  measure  of  perfonnance  in  the  physical  domain  for  the  satellites  in 
the  Kosovo  scenario,  but  didn’t  seem  as  applicable  for  measuring  the  perfonnance  of 
other  agents,  such  as  the  JSTARS  and  GlobalHawk.  Analysis  of  the  infonnation  domain 
provided  different  approaches  and  ways  of  looking  at  the  average  number  of  messages 
being  handled  by  the  network  for  various  communication  channels.  The  metric  of 
average  channel  message  load  seemed  to  be  a  suitable  measure  for  some  channels,  such 
as  for  TacAirQ_Sit,  but  not  as  suitable  a  measure  of  perfonnance  for  other  channels,  such 
as  JSTARSQ  Sit.  Utilization  of  Killer  Victim  Scoreboard  (KVS)  information  for 
measuring  outcomes  pertaining  to  the  cognitive  domain  seemed  to  be  the  most  consistent 
and  reliable  measure,  as  compared  with  the  measures  for  the  physical  and  infonnation 
domains.  Positive  trends  for  the  Blue  Force  were  seen  in  comparing  the  case  of  no 
degradation  effects  to  those  three  cases  employing  effects  that  would  degrade 
performance  of  the  sensors  and  communication  devices.  In  the  no  effects  case  as 
compared  with  the  three  cases  of  varying  degradation,  Blue  killed  more  Red  and  spared 
more  Brown  agents.  An  increase  of  Kosovar  houses  killed  in  the  full  effects  degradation 
case  was  found  to  be  statistically  significant. 
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V.  Conclusions 


5.1  Overview 

This  research  utilized  a  SEAS  scenario  representative  of  the  Kosovo  conflict 
during  1999  to  simulate  the  perfonnance  of  an  NCW  enabled  force  and  to  provide  an 
investigative  framework  from  which  to  identify  appropriate  measures  of  NCW  that  are 
available  from  the  SEAS  software.  The  methodology  for  determining  appropriate  NCW 
measures  was  conducted  by  means  of  implementing  effects  within  the  Kosovo  scenario 
which  degraded  the  sensing  and  communications  ability  of  the  Blue  Force.  This  chapter 
presents  a  summary  of  the  conclusions  drawn  from  statistical  output  analysis  conducted 
for  measures  of  the  three  domains  of  NCW  for  a  baseline  case  of  the  Kosovo  scenario 
with  no  degradation  effects  versus  three  cases  possessing  varying  levels  of  degradation 
effects.  Following  a  description  of  conclusions  drawn  from  analysis,  recommendations 
for  improvement  of  SEAS  software  and  the  SEAS  Post  Processor  are  addressed.  This 
chapter  concludes  with  several  suggestions  for  future  research. 

5.2  Analysis  Conclusions  and  Limitations 

In  general,  the  physical  domain  measure  of  average  detection  distance  was  found 
to  be  an  appropriate  measure  for  the  Blue  Force  satellites  in  the  Kosovo  scenario,  but  not 
for  other  agents  affected  by  the  degradation  effects,  namely  the  JSTARS  and 
GlobalHawk.  The  analysis  was  somewhat  limited  by  the  fact  that  determining  the 
number  of  detections  for  various  platforms  was  very  challenging  and  intensive,  especially 
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when  dealing  with  the  considerably  large  sensor  detection  output  files  from  30  runs  of  the 
scenario. 

For  the  information  domain,  measuring  various  averages  of  communication 
channel  loading  for  single  run  as  well  as  for  multiple  runs  provided  some  insight  into  the 
affect  of  degradation  on  the  active  communications  channels  in  the  Kosovo  scenario. 
Message  count  data  for  all  channels  in  the  Kosovo  scenario  was  analyzed  from  three 
different  perspectives:  total  count  and  average  message  load  numbers  for  eleven  runs  of 
the  scenario,  total  count  and  average  active  message  load  for  all  channels  handling 
messages  in  the  scenario  (some  channels  saw  no  activity),  and  average  message  loading 
for  the  top  four  active  channels  plotted  according  to  ten  ten-hour  segments  of  one 
simulation  run.  Encouraging  trends  were  observed  from  analysis  of  the  average  message 
loading  measure,  such  as  a  reduction  in  the  average  message  activity  for  Blue’s  primary 
channel,  TacAirQ_Sit,  from  the  baseline  no  degradation  effects  case  to  the  full 
degradation  effects.  Also,  message  loads  for  the  top  four  active  channels  in  the  scenario 
were  seen  to  approximately  match  the  two  phases  of  war  pattern,  phases  which  were 
initially  discovered  by  Destefano  in  his  analysis  of  kill  data  for  one  run  of  the  Kosovo 
scenario.  Analysis  of  the  JSTARS  target  sighting  channel  showed  an  unexpected  increase 
in  average  message  load  for  the  degraded  case,  which  slightly  undennined  the  legitimacy 
of  the  chosen  technique  for  measuring  communication  channel  perfonnance.  However, 
this  unexpected  result  may  be  unique  to  the  JSTARS  platform  in  this  scenario  and  does 
not  necessarily  totally  invalidate  the  technique. 

The  most  compelling  results  were  seen  in  analysis  of  kill  data  for  the  cognitive 
domain.  Both  number  of  kills  and  number  of  victims  for  the  Blue,  Red,  and  Brown 
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forces  were  analyzed  in  order  to  determine  what  affect  the  various  degradation  states 
would  have  on  the  attrition  numbers.  Clear  trends  were  seen  portraying  fewer  kills  for 
Blue  and  higher  losses  for  Brown  in  the  degraded  cases  versus  the  baseline  case,  all  of 
which  were  compelling  because  these  are  the  results  that  one  would  logically  expect  to 
observe  given  the  primary  role  played  by  sensors  and  communications  devices  in  the 
scenario.  These  killer  and  victim  outcomes  give  a  fair  representation  of  Blue’s  reliance 
on  the  network  and  show  how  Blue’s  perfonnance  suffers  when  the  network  capability  is 
reduced. 

5.3  Recommendations  for  SEAS  Improvement 

The  Excel-based  SEAS  Post  Processor  is  a  very  useful  tool  for  processing, 
fdtering,  and  graphically  representing  various  types  of  output  data  created  by  a  typical 
SEAS  scenario.  However,  several  problems  and  limitations  were  encountered  over  the 
course  of  analysis  for  this  effort,  especially  when  analyzing  output  data  for  the  various 
communications  channels  in  the  Kosovo  scenario.  The  standard  plots  for  output  data  for 
the  various  communications  channels  that  were  available  from  the  SEAS  Post  Processor 
ultimately  proved  to  be  more  confusing  than  they  were  useful.  There  is  considerable 
room  for  improvement  as  far  as  the  options  available  for  filtering  and  setting  up  plots  for 
communications  data.  Also,  the  lack  of  ability  to  quickly  filter  and  process  large  data 
files  was  a  major  hindrance  in  using  SEAS  Post  Processor  throughout  the  analysis 
process.  This  was  especially  true  for  the  communications  data  analysis.  One 
communications  data  output  file  from  30  runs  of  the  Kosovo  scenario  of  moderate  file 
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size  relative  to  the  sensor  data  output  files  (typically  70  MB  for  the  communications  files 
versus  375  MB  for  the  sensor  files)  required  one  hour  just  to  load  within  Post  Processor. 

5.4  Future  Research 

There  are  several  follow-on  activities  that  could  be  pursued  in  order  to  enhance 
and  build  upon  the  findings  of  this  research.  The  utilization  of  write  statements  is  a 
SEAS  coding  technique  which  offers  great  promise  and  potential  in  its  ability  to  extract 
specific  pieces  of  data.  Employing  write  statements,  either  in  the  Kosovo  scenario  or  in 
another  SEAS  scenario  which  adequately  represents  NCW,  could  potentially  provide  a 
powerful  aid  in  helping  to  filter  and  isolate  appropriate  NCW  measures,  especially  in  the 
information  and  cognitive  domains.  A  few  measures  which  write  statements  may  be  able 
to  capture  include  tracking  the  overall  cycle  time  required  to  detect  and  neutralize  a 
target  and  tracking  the  number  of  target  sighting  messages  from  a  specific  key  sensor  that 
are  being  relayed  to  a  certain  weapons  platfonn. 

Another  research  methodology  that  could  prove  to  be  a  useful  approach  in  finding 
appropriate  measures  for  NCW  is  an  analysis  of  the  outcomes  resulting  from  changing 
the  degree  of  information  sharing  by  varying  the  message  cueing  attribute  for  key  agents 
in  a  scenario.  Measures  of  performance  could  be  analyzed  relative  to  the  extent  to  which 
agents  in  the  scenario  are  sharing  information. 

Network  Centric  Warfare  is  a  continually  evolving  concept.  Research  and 
analysis  of  appropriate  measures  is  likely  to  be  an  ongoing  activity  for  as  long  as  the 
DoD  includes  NCW  as  part  of  its  military  doctrine  and  strategy.  Continued  pursuit  of 
understanding  NCW  and  how  to  appropriately  measure  it  through  use  of  combat  models, 
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simulations,  case  studies,  and  lessons  learned  from  practical  experience  will  definitely 
continue  to  benefit  our  forces  and  improve  their  current  and  future  efficiency  of 
operations  in  the  brave  new  network-centric  environment. 
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Appendix  A.  Paired-t  Tests  Data 


Average  Detection  Distances  for  All  Four  Cases 


Blue.USAFEUROPE.Sat1#1.B  Sat  EO  Sensor 

No  Effects 

Full  Effects 

z 

Z  bar 

(Z-Z  bar)A2 

Var(Z  bar) 

1.00 

1280.99 

1142.02 

frMMklil 

LSHI 

2.00 

1292.78 

1082.57 

1018.32 

1079.37 

264.39 

4.00 

1267.11 

1079.11 

94.21 

5.00 

1279.05 

1100.92 

0.03 

1081.55 

75.65 

7.00 

1287.16 

1079.83 

842.70 

8.00 

1081.38 

1127.17 

606.04 

10.00 

1263.23 

1060.65 

589.34 

11.00 

1279.33 

1096.14 

23.83 

1058.44 

496.92 

13.00 

1258.70 

1193.13 

12708.24 

14.00 

1272.67 

1100.41 

36.52 

1063.20 

1800.48 

16.00 

1283.66 

1089.07 

265.10 

17.00 

1293.64 

1084.06 

978.68 

U£mkhi3EE] 

1100.33 

804.24 

19.00 

1240.94 

20.00 

1271.79 

1074.53 

15.68 

22.00 

1239.56 

1116.96 

3102.49 

23.00 

1269.47 

1089.76 

2.00 

1079.51 

248.32 

25.00 

1248.26 

1072.57 

6.81 

26.00 

1234.35 

1094.47 

1476.12 

1082.29 

8.60 

1095.58 

30.67 

1086.57 

525.73 

1076.04 

78.20 

Averages: 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

166.38  190.23 


|Blue.USAFEUROPE.Sat1#1.B  Sat  EO  Sensor 

KB,.rr  m 

Terrain  Only 

z  | 

Z  bar 

(Z-Z  bar)A2 

1091.97 

208.84 

18.10| 

2.00 

1292.78 

193.15 

345.37 

3.00 

1273.93 

1088.26 

185.67 

123.31 

4.00 

1267.11 

1084.00 

183.12 

73.11 

5.00 

1279.05 

1089.57 

189.48 

222.22 

6.00 

1251.16 

1084.54 

166.62 

63.25 

7.00 

1287.16 

1096.40 

190.75 

261.93 

8.00 

1301.90 

1104.75 

197.15 

510.06 

9.00 

1280.85 

1064.16 

216.69 

1774.24 

10.00 

1263.23 

1104.90 

158.33 

263.59 

11.00 

1279.33 

1129.70 

149.63 

622.01 

12.00 

1259.03 

1081.26 

177.77 

10.23 

13.00 

1258.70 

1099.59 

159.11 

238.96 

14.00 

1272.67 

1084.30 

188.37 

190.54 

15.00 

1283.93 

1084.59 

199.34 

613.62 

16.00 

1283.66 

1153.34 

130.32 

1958.17 

17.00 

1293.64 

1105.61 

188.03 

181.17 

18.00 

1306.99 

1101.26 

205.73 

971.22 

19.00 

1240.94 

1105.64 

135.30 

1542.18 

20.00 

1271.79 

1081.27 

190.52 

254.42 

21.00 

1256.79 

1080.51 

176.27 

2.91 

22.00 

1239.56 

1076.77 

162.79 

138.64 

23.00 

1269.47 

1072.20 

197.27 

515.19 

24.00 

1242.06 

1091.44 

150.61 

573.84 

25.00 

1248.26 

1118.63 

129.63 

2019.49 

26.00 

1234.35 

1083.18 

151.16 

547.74 

27.00 

1257.66 

1096.35 

161.31 

175.91 

28.00 

1279.42 

1089.36 

190.06 

239.84 

29.00 

1287.80 

1106.07 

181.73 

51.34 

30.00 

1245.50 

1103.37 

142.12 

1052.82 

Averages: 

1269.66 

1095.09 

SUM= 

15746.15 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

165.88  183.26 
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95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

14.09  37.98 


95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

164.02  189.30 
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|  Blue.USAFEUROPE.Sat1#2.B  Sat  EO  Sensor 

Terrain  Only 

z  | 

Z  bar 

(Z-Z  bar)A2 

ISHE&ifZI 

1098.29 

nMii  f  \  i 

M  'Ti'i| 

807.62 

20.96| 

1108.86 

176.69 

3.00 

1274.63 

1110.91 

163.73 

0.07 

4.00 

1265.26 

1078.71 

186.55 

508.36 

5.00 

1283.17 

1111.50 

171.67 

58.91 

6.00 

1289.71 

1122.67 

167.04 

9.27 

7.00 

1307.49 

1125.85 

181.63 

311.04 

8.00 

1236.81 

1116.70 

120.11 

1925.89 

9.00 

1289.30 

1110.70 

178.60 

213.28 

10.00 

1248.63 

1129.70 

118.92 

2031.76 

11.00 

1285.04 

1131.82 

153.22 

116.13 

12.00 

1262.96 

1128.44 

134.52 

869.20 

13.00 

1295.55 

1113.66 

181.89 

320.01 

14.00 

1294.60 

1135.69 

158.92 

25.84 

15.00 

1290.07 

1122.06 

168.01 

16.12 

16.00 

1285.23 

1179.46 

105.77 

3390.04 

17.00 

1248.79 

1110.12 

138.67 

641.39 

18.00 

1307.33 

1130.70 

176.63 

159.65 

19.00 

1261.66 

1100.54 

161.11 

8.32 

20.00 

1284.45 

1111.35 

173.10 

82.86 

21.00 

1302.29 

1100.54 

201.75 

1425.27 

22.00 

1284.47 

1103.09 

181.38 

302.24 

23.00 

1266.88 

1120.88 

146.01 

323.73 

24.00 

1297.42 

1118.46 

178.96 

223.77 

25.00 

1278.43 

1108.97 

169.46 

29.84 

26.00 

1250.15 

1119.14 

131.01 

1088.16 

27.00 

1302.10 

1092.02 

210.08 

2123.52 

28.00 

1272.96 

1093.85 

179.11 

228.44 

29.00 

1304.11 

1115.12 

188.99 

624.62 

30.00 

1273.82 

1123.86 

149.96 

196.97 

Averages: 

1279.79 

1115.79 

SUM= 

18239.01 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

154.65  173.35 


|  Blue.USAFEUROPE.Sat1#2.B  Sat  EO  Sensor 

Run  # 

No  Effects 

Weather  Only 

Z 

Z  bar 

(Z-Z  bar)A2 

1.00 

1290.71 

1229.25 

61.46 

18.05 

1883.83 

2.00 

1259.56 

1225.54 

34.02 

254.95 

3.00 

1274.63 

1259.52 

15.11 

8.64 

4.00 

1265.26 

1360.17 

-94.92 

12762.03 

5.00 

1283.17 

1286.88 

-3.71 

473.83 

6.00 

1289.71 

1309.83 

-20.11 

1456.68 

7.00 

1307.49 

1311.32 

-3.84 

479.14 

8.00 

1236.81 

1260.68 

-23.87 

1757.63 

9.00 

1289.30 

1280.22 

9.08 

80.57 

10.00 

1248.63 

1238.52 

10.10 

63.24 

11.00 

1285.04 

1178.53 

106.51 

7824.85 

12.00 

1262.96 

1210.43 

52.53 

1188.77 

13.00 

1295.55 

1311.74 

-16.19 

1172.57 

14.00 

1294.60 

1285.56 

9.04 

81.25 

15.00 

1290.07 

1345.33 

-55.26 

5374.35 

16.00 

1285.23 

1248.59 

36.65 

345.65 

17.00 

1248.79 

1202.18 

46.62 

815.80 

18.00 

1307.33 

1174.49 

132.85 

13177.17 

19.00 

1261.66 

1305.18 

-43.53 

3792.04 

20.00 

1284.45 

1270.24 

14.21 

14.77 

21.00 

1302.29 

1278.56 

23.73 

32.26 

22.00 

1284.47 

1262.97 

21.50 

11.89 

23.00 

1266.88 

1238.10 

28.78 

115.09 

24.00 

1297.42 

1187.21 

110.20 

8491.51 

25.00 

1278.43 

1254.24 

24.19 

37.69 

26.00 

1250.15 

1285.82 

-35.67 

2886.22 

27.00 

1302.10 

1273.19 

28.92 

118.03 

28.00 

1272.96 

1231.72 

41.24 

537.64 

29.00 

1304.11 

1281.47 

22.64 

21.01 

30.00 

1273.82 

1264.50 

9.32 

76.28 

Averages: 

1279.79 

1261.73 

SUM= 

65335.36 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

0.36  35.75 
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Average  Detection  Distances  for  All  Four  Cases 


|Blue.USAFEUROPE.USAF  CAOC#1.F15  SEADSqdn#5.F15E#1.AC  Elint  ] 

1 

Z 

Z  bar 

(Z-Z  bar)A2 

30.96 

-0.44 

263.83 

7.93 1 

59.04 

5.07 

30.34 
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95%TestStat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-6.19  5.31 


Blue.USAFEUROPE.USAF_CAOC#1  .FI  5_SEADSqdn#5.F1 5E#1  .AC_Elint 


Terrain  Only 
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SUM= 

5526.77 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-11.71  -1.41 
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95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-4.75  7.87 


Average  Detection  Distances  for  All  Four  Cases 

Blue.USAFEUROPE.USAF  CAOC#1.F15  SEADSqdn#5.F15E#4.AC  Elint 


95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-8.22  8.97 
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Blue.USAFEUR0PE.USAF_CA0C#1  .FI  5_SEADSqdn#5.F1 5E#4.AC_Elint 
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Averages: 
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SUM= 
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95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-8.45  4.16 


Blue.USAFEUROPE.USAF_CAOC#1.F15_SEADSqdn#5.F15E#4.AC_Elint 


No  Effects 

Weather  Only 
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95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-2.83  8.97 
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Average  Detection  Distances  for  All  Four  Cases 


Average  Detection  Distance  Comparisons  for  All  6  F-15  ACEIint  Sensors 
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Average  Detection  Distance  Comparisons  for  All  6  F-15  AC  EIint  Sensors 
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SUM= 
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95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-7.05  0.91 
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Average  Detection  Distance  Comparisons  for  All  6  F-15  AC  EIint  Sensors 


No  Effects 
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|^E21S9 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-2.99  4.23 


Average  Detection  Distances  for  All  Four  Cases 
Blue.USAFEUROPE.USAF_CAOC#1.JSTARS#1.JSTARS_MTI 


|  Run  # 

|  No  Effects  | 

Full  Effects 
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70.86 
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59.36 

64.91 

-5.56 

33.28 

19.00 

70.36 

56.48 

13.88 

186.86 

20.00 

67.95 

66.57 

1.38 

1.38 

21.00 

66.59 

61.42 

5.17 

24.62 

22.00 

62.84 

65.41 

-2.57 

7.72 

23.00 

59.04 

66.62 

-7.58 

60.60 

24.00 

64.08 

65.43 

-1.34 

2.41 

25.00 

65.82 

62.02 

3.79 

12.83 

26.00 

70.29 

70.15 

0.14 

0.00 

27.00 

66.36 

64.64 

1.72 

2.29 

28.00 

60.89 

63.77 

-2.89 

9.58 

29.00 

61.04 

60.72 

0.32 

0.01 

30.00 

58.02 

59.47 

-1.46 

2.78 

Averages: 

64.05 

63.84 

SUM= 

1099.98 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-2.09  2.51 
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Blue.USAFEUR0PE.USAF_CA0C#1  .JSTARS#1  JSTARS_MTI 


Terrain  Only 

Z 

Z  bar 

(Z-Z  bar)A2 
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68.11 
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64.68 
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68.71 

63.60 
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23.28 

5.00 

65.08 

69.10 

-4.02 

18.54 

6.00 

57.99 

72.78 

-14.80 

227.45 

7.00 

59.36 

62.09 

-2.74 

9.12 

8.00 

60.93 

73.28 

-12.36 

159.74 

9.00 

62.85 

52.48 

10.37 

101.66 

10.00 

59.76 

68.18 

-8.42 

75.72 

11.00 

68.20 

66.29 

1.91 

2.65 

12.00 

59.18 

57.64 

1.54 

1.58 

13.00 

66.24 

61.96 

4.29 

16.03 

14.00 

66.01 

55.42 

10.59 

106.30 

15.00 

68.69 

61.88 

6.82 

42.67 

16.00 

70.86 

58.69 

12.17 

141.29 

17.00 

62.49 

59.51 

2.98 

7.26 

18.00 

59.36 

60.88 

-1.52 

3.26 

19.00 

70.36 

67.00 

3.36 

9.49 

20.00 

67.95 

63.25 

4.70 

19.47 

21.00 

66.59 

65.69 

0.90 

0.38 

22.00 

62.84 

57.95 

4.89 

21.22 

23.00 

59.04 

66.87 

-7.83 

65.82 

24.00 

64.08 

66.53 

-2.44 

7.43 

25.00 

65.82 

61.46 

4.36 

16.57 

26.00 

70.29 

61.85 

8.44 

66.48 

27.00 

66.36 

68.70 

-2.34 

6.88 

28.00 

60.89 

74.46 

-13.57 

191.92 

29.00 

61.04 

64.54 

-3.49 

14.27 

30.00 

58.02 

58.32 

-0.30 

0.34 

Averages: 

64.05 

63.77 

SUM= 

1593.04 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-2.48  3.05 


Blue. USAFEU  ROPE.  USAF_CAOC#1  .JSTARS#1  .JSTARS_MTI 


Weather  Only 

Z 

Z  bar 

(Z-Z  bar)A2 

Var(Z  bar) 
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2.53 
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3.00 
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0.32 
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4.00 

68.71 

70.36 

-1.65 

1.03 

60.69 

4.38 

25.16 

6.00 

57.99 

54.20 

3.79 

19.51 

7.00 

59.36 

77.92 

-18.56 

321.58 

63.71 

-2.78 

4.62 

9.00 

62.85 

60.51 

2.34 

8.84 

10.00 

59.76 

64.74 

-4.98 

18.94 

74.00 

-5.79 

26.66 

12.00 

59.18 

56.62 

2.56 

10.17 

13.00 

66.24 

64.49 

1.75 

5.68 

67.42 

-1.40 

0.60 

HIHHBHHHBBHHIlHEZsIsflHBHHHHflHfllliHHHI 

69.98 

-1.28 

0.43 

59.89 

10.97 

134.58 

67.41 

-4.92 

18.39 

70.41 

-11.06 

108.74 

72.20 

-1.84 

1.45 

57.87 

10.08 

114.75 

67.80 

-1.21 

0.34 

59.19 

3.64 

18.29 

63.21 

-4.17 

12.50 

67.66 

-3.58 

8.69 

78.69 

-12.88 

72.32 

-2.03 

1.95 

65.68 

0.69 

1.73 

63.37 

-2.48 

3.41 

29.00 

61.04 

57.89 

3.15 

14.29 

30.00 

58.02 

54.86 

3.16 

14.36 

Averages: 

64.05 

64.68 

1225.04 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-3.05  1.79 
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Average  Detection  Distances  for  All  Four  Cases 


Blue.USAFEUROPE.USAF_CAOC#1.GlobalHawk#2.GH_EO_SAR 


Z 

Z  bar 

(Z-Z  bar)A2 

Var(Z  bar) 

24.67 

0.03 

-0.06 

0.01 

25.02 

-0.09 

0.00 

3.00 

24.98 

25.01 

-0.03 

0.00 

4.00 

24.55 

24.47 

0.08 

0.02 

WM  1 1  li 

24.58 

0.08 

0.02 

6.00 

24.80 

25.12 

-0.31 

0.06 

7.00 

24.79 

24.93 

-0.14 

0.01 

24.95 

-0.28 

0.05 

25.10 

-0.32 

0.06 

24.74 

-0.12 

0.00 

24.21 

0.69 

0.56 

12.00 

24.80 

24.61 

0.19 

0.06 

13.00 

24.69 

24.43 

0.26 

0.10 

24.96 

-0.24 

0.03 

15.00 

24.98 

24.92 

0.05 

0.01 

16.00 

24.69 

24.90 

-0.21 

0.02 

M  f  II  M 

24.65 

-0.02 

0.00 

18.00 

24.89 

24.74 

0.15 

0.04 

19.00 

24.57 

25.15 

-0.58 

0.26 

25.01 

-0.40 

0.11 

21.00 

24.96 

24.96 

0.00 

0.00 

22.00 

24.62 

25.03 

-0.41 

0.12 

25.09 

-0.50 

0.19 

24.00 

24.79 

24.67 

0.12 

0.03 

25.00 

24.78 

24.73 

0.06 

0.01 

24.93 

-0.14 

0.01 

27.00 

24.66 

-0.36 

0.09 

28.00 

24.81 

0.47 

0.29 

24.50 

0.46 

0.27 

30.00 

24.79 

25.20 

-0.41 

0.12 

Averages: 

24.76 

24.82 

2.59 

95%TestStat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-0.18  0.05 


Blue.USAFEUROPE.USAF_CAOC#1.GlobalHawk#2.GH_EO_SAR 


Terrain  Only 

Z 

Z  bar 

(Z-Z  bar)A2 

1.00 

24.70 

24.83 

-0.13 

-0.06 

0.01 

0.00 

2.00 

24.93 

24.77 

0.15 

0.05 

3.00 

24.98 

24.94 

0.04 

0.01 

4.00 

24.55 

24.84 

-0.29 

0.05 

5.00 

24.67 

24.87 

-0.20 

0.02 

6.00 

24.80 

24.80 

0.00 

0.00 

7.00 

24.79 

24.89 

-0.09 

0.00 

8.00 

24.67 

24.77 

-0.10 

0.00 

9.00 

24.78 

24.88 

-0.10 

0.00 

10.00 

24.62 

24.64 

-0.02 

0.00 

11.00 

24.90 

24.78 

0.12 

0.03 

12.00 

24.80 

24.83 

-0.03 

0.00 

13.00 

24.69 

24.71 

-0.03 

0.00 

14.00 

24.72 

24.80 

-0.07 

0.00 

15.00 

24.98 

24.77 

0.21 

0.07 

16.00 

24.69 

24.75 

-0.06 

0.00 

17.00 

24.63 

24.85 

-0.22 

0.03 

18.00 

24.89 

24.63 

0.26 

0.10 

19.00 

24.57 

24.90 

-0.33 

0.07 

20.00 

24.61 

25.08 

-0.47 

0.17 

21.00 

24.96 

24.95 

0.02 

0.01 

22.00 

24.62 

24.65 

-0.03 

0.00 

23.00 

24.59 

24.79 

-0.20 

0.02 

24.00 

24.79 

24.64 

0.15 

0.04 

25.00 

24.78 

24.74 

0.05 

0.01 

26.00 

24.79 

24.85 

-0.06 

0.00 

27.00 

24.66 

25.04 

-0.37 

0.10 

28.00 

24.81 

24.77 

0.04 

0.01 

29.00 

24.96 

24.83 

0.13 

0.04 

30.00 

24.79 

24.91 

-0.11 

0.00 

Averages: 

24.76 

24.82 

SUM= 

0.85 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-0.12  0.00 
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95%  Test  Stat:  1 .70 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

0.03  0.23 


Kill  and  Victim  Data  Analysis _ 

Red  SA  and  Radar  Vans  Victim  Counts  -  TOTAL  per  run 


95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-0.32  3.05 
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Red  SA  and  Radar  Vans  Victim  Counts  -  TOTAL  per  run 


Terrain  Only 

Z 

Z  bar 

(Z-Z  bar)A2 

19 

2.00 

0.13 

4.24 

0.44 

2.00 

24 

18 

36.71 

3.00 

13 

24 

-11.00 

119.70 

4.00 

24 

24 

0.00 

0.00 

5.00 

24 

22 

2.00 

4.24 

6.00 

24 

24 

0.00 

0.00 

7.00 

24 

24 

0.00 

0.00 

8.00 

22 

22 

0.00 

0.00 

9.00 

22 

21 

1.00 

1.12 

10.00 

24 

24 

0.00 

0.00 

11.00 

24 

24 

0.00 

0.00 

12.00 

23 

23 

0.00 

0.00 

13.00 

22 

23 

-1.00 

0.89 

14.00 

24 

17 

7.00 

49.83 

15.00 

19 

24 

-5.00 

24.41 

16.00 

24 

18 

6.00 

36.71 

17.00 

24 

24 

0.00 

0.00 

18.00 

22 

24 

-2.00 

3.77 

19.00 

20 

20 

0.00 

0.00 

20.00 

24 

21 

3.00 

9.36 

21.00 

22 

22 

0.00 

0.00 

22.00 

24 

22 

2.00 

4.24 

23.00 

22 

23 

-1.00 

0.89 

24.00 

24 

23 

1.00 

1.12 

25.00 

23 

22 

1.00 

1.12 

26.00 

21 

21 

0.00 

0.00 

27.00 

21 

24 

-3.00 

8.65 

28.00 

24 

21 

3.00 

9.36 

29.00 

16 

24 

-8.00 

63.06 

30.00 

24 

23 

1.00 

1.12 

Averages: 

22.30 

22.17 

SUM= 

380.58 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-1.22  1.48 


Red  SA  and  Radar  Vans  Victim  Counts  -  TOTAL  per  run 


Run  # 

No  Effects 

Weather  Only 

Z 

Z  bar 

(Z-Z  bar)A2 

vmwt. ni 

■  i  BEil 

20 

1.00 

0.83 

1.00 

0.70 

2.00 

24 

24 

0.00 

0.00 

3.00 

13 

24 

-11.00 

121.00 

19 

5.00 

25.00 

5.00 

24 

24 

0.00 

0.00 

6.00 

24 

16 

8.00 

64.00 

24 

0.00 

0.00 

21 

1.00 

1.00 

24 

-2.00 

4.00 

10.00 

24 

19 

5.00 

11.00 

24 

24 

0.00 

flE2L; 

21 

2.00 

4.00 

13.00 

22 

19 

3.00 

9.00 

14.00 

24 

16 

8.00 

64.00 

Q  8  KSBED 

20 

-1.00 

1.00 

16.00 

24 

23 

1.00 

1.00 

17.00 

24 

23 

1.00 

1.00 

24 

-2.00 

4.00 

19.00 

20 

24 

-4.00 

16.00 

20.00 

24 

18 

6.00 

36.00 

24 

-2.00 

4.00 

22 

2.00 

4.00 

21 

1.00 

1.00 

24.00 

24 

11 

13.00 

169.00 

25.00 

23 

24 

-1.00 

1.00 

- 

24 

-3.00 

9.00 

27.00 

21 

23 

-2.00 

4.00 

28.00 

24 

22 

2.00 

4.00 

22 

-6.00 

36.00 

30.00 

24 

24 

0.00 

0.00 

Averages: 

22.30 

21.47 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-0.88  2.54 
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Kill  and  Victim  Data  Analysis 


Brown  Houses  Victim  Counts  -  TOTAL  per  run 


Z 

Z  bar 

(Z-Z  bar)A2 

3.00 

-2.20 

9.00 

1.08 

1.00 

1.00 

25.00 

-6.00 

36.00 

-2.00 

4.00 

6.00 

7 

3.00 

9.00 

7.00 

6 

-1.00 

1.00 

0.00 

0.00 

3.00 

9.00 

-2.00 

1.00 

3.00 

9.00 

13.00 

8 

1.00 

1.00 

9.00 

15.00 

2 

8 

-6.00 

36.00 

16.00 

7 

6 

1.00 

1.00 

-2.00 

4.00 

4.00 

-2.00 

4.00 

-1.00 

1.00 

-2.00 

4.00 

0.00 

-1.00 

1.00 

24.00 

6 

7 

1.00 

25.00 

6 

5 

1.00 

6 

-6.00 

36.00 

27.00 

0.00 

9 

-9.00 

81.00 

28.00 

0.00 

5 

-5.00 

25.00 

7 

-7.00 

49.00 

24 

-24.00 

576.00 

5.37 

7.57 

942.00 

95%TestStat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-4.33  -0.07 


Brown  Houses  Victim  Counts  -  TOTAL  per  run 


Terrain  Only 

Z 

Z  bar 

(Z-Z  bar)A2 

Enasaa 

5 

3.00 

-0.40 

9.36 

0.29 

2.00 

10 

7 

9.36 

3.00 

6 

7 

-1.00 

0.89 

4.00 

2 

10 

-8.00 

63.06 

5.00 

7 

4 

3.00 

9.36 

6.00 

7 

7 

0.00 

0.00 

7.00 

6 

6 

0.00 

0.00 

8.00 

8 

8 

0.00 

0.00 

9.00 

7 

7 

0.00 

0.00 

10.00 

6 

7 

-1.00 

0.89 

11.00 

8 

5 

3.00 

9.36 

12.00 

10 

6 

4.00 

16.48 

13.00 

8 

9 

-1.00 

0.89 

14.00 

4 

7 

-3.00 

8.65 

15.00 

2 

5 

-3.00 

8.65 

16.00 

7 

7 

0.00 

0.00 

17.00 

1 

5 

-4.00 

15.53 

18.00 

9 

11 

-2.00 

3.77 

19.00 

4 

8 

-4.00 

15.53 

20.00 

7 

7 

0.00 

0.00 

21.00 

8 

5 

3.00 

9.36 

22.00 

7 

3 

4.00 

16.48 

23.00 

7 

5 

2.00 

4.24 

24.00 

6 

5 

1.00 

1.12 

25.00 

6 

6 

0.00 

0.00 

26.00 

0.00 

3 

-3.00 

8.65 

27.00 

0.00 

2 

-2.00 

3.77 

28.00 

0.00 

6 

-6.00 

35.30 

29.00 

0.00 

0 

0.00 

0.00 

30.00 

0.00 

0 

0.00 

0.00 

Averages: 

5.37 

5.77 

SUM= 

250.69 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-1.50  0.70 
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Brown  Houses  Victim  Counts  -  TOTAL  per  run 


No  Effects 

Weather  Only 

Z 

Z  bar 

msmsM 

EB35EB1 

5 

3.00 

-0.43 

9.00 

QUAKES 

8 

2.00 

4.00 

3.00 

6 

6 

0.00 

0.00 

4.00 

2 

8 

-6.00 

36.00 

9 

-2.00 

4.00 

4 

3.00 

9.00 

7 

-1.00 

1.00 

3 

5.00 

25.00 

9.00 

7 

7 

0.00 

0.00 

10.00 

6 

6 

0.00 

0.00 

1  B 

8 

0.00 

0.00 

12.00 

10 

6 

4.00 

16.00 

13.00 

8 

8 

0.00 

0.00 

6 

-2.00 

4.00 

7 

-5.00 

25.00 

11 

-4.00 

16.00 

-8.00 

64.00 

7 

2.00 

4.00 

7 

-3.00 

9.00 

6 

1.00 

1.00 

1 

7.00 

49.00 

5 

2.00 

4.00 

5 

2.00 

4.00 

6 

0.00 

0.00 

7 

-1.00 

1.00 

26.00 

0.00 

5 

-5.00 

25.00 

27.00 

0.00 

7 

-7.00 

49.00 

0 

0.00 

0.00 

0 

0.00 

0.00 

0 

0.00 

0.00 

Averages: 

5.37 

5.80 

2^^EZ3Zil!] 

95%  Test  Stat:  2.04 

Confidence  Interval:  W  bar  +/-  Test  Stat*SQRT(Var(W  Bar))  = 


Lower  Limit  Upper  Limit 

-1.75  0.88 
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